A method, a computer program product, and a computer system for determining a location using street view images. A mobile device obtains an image captured by the mobile device, obtains a set of street view images, and obtains a first graph of the image captured by the mobile device and a plurality of second graphs of the set of the street view images. The first graph includes nodes representing interest points in the image captured by the mobile device, and the plurality of the second graphs includes nodes representing interest points in the set of the street view images. The mobile device determines a location of the mobile device, based on relevance between the first graph and the plurality of the second graphs.