Harlem is located in Upper Manhattan, often referred to as Uptown by locals. It stretches from the Harlem River and East River in the east, to the Hudson River to the west; and between 155th Street in the north, where it meets Washington Heights, and an uneven boundary along the south that runs along either 96th Street east of Fifth Avenue or 110th Street west of Fifth Avenue.

Central Harlem is bounded by Fifth Avenue on the east, Central Park on the south, Morningside Park, St. Nicholas Avenue and Edgecombe Avenue on the west, and the Harlem River on the north. A chain of three large linear parks—Morningside Park, St. Nicholas Park and Jackie Robinson Park—are situated on steeply rising banks and form most of the district's western boundary. On the east, Fifth Avenue and Marcus Garvey Park, also known as Mount Morris Park, separate this area from East Harlem.