Video surveillance systems increasingly use H.264 coding to achieve 24 x 7 recording and streaming. However, with the proliferation of security cameras, and the need to store several months of video, bandwidth and storage costs can be significant. We propose a new compression technique to significantly improve the coding efficiency of H.264 for surveillance video. Video content is analyzed and video semantics are extracted using video analytics algorithms such as segmentation, classification and tracking. In contrast to existing approaches, our Analytics-Modulated Compression (AMC) scheme does not require coding of object shape information and produces bit-streams that are standards-compliant and not limited to specific H.264 profiles. Extensive experiments were conducted involving real surveillance scenes. Results show that our technique achieves compression gains of 67% over JM. We also introduced AMC Rate Control (AMC RC) which allocates bits in response to scene dynamics. AMC RC is shown to significantly reduce artifacts in constant-bitrate video at low bitrates.