Fort Point was constructed by the U.S. Army Corps of Engineers between 1853 and 1861 to prevent entrance of a hostile fleet into San Francisco Bay.

The fort was designed to mount 126 massive cannon. Rushed to completion at the beginning of the Civil War, Fort Point was first garrisoned in February of 1861 by Company I, 3rd U.S. Artillery Regiment. The fort was occupied throughout the Civil War, but the advent of faster, more powerful rifled cannon made brick forts such as Fort Point obsolete. In 1886 the troops were withdrawn, and the last cannon were removed about 1900. The fort was then used for storage and training purposes for many years.

Between 1933 and 1937 the fort was used as a base of operations for the construction of the Golden Gate Bridge. During World War II, Fort Point was occupied by about 100 soldiers who manned searchlights and rapid-fire cannon mounted atop the fort as part of the protection of a submarine net strung across the entrance to the Bay.

Fort Point is the only third system brick fort on the west coast of the United States. It became a National Historic Site on October 16th, 1970.