forums.tropy.org

Use binary exif tag value in GPS Longitude & latitude

I notice that Tropy imports an odd value for the exif GPS longitude & latitude.

It seems that it takes the text version and strips out all characters except for numbers, commas, and periods. For example

“74 deg 1’ 44.23” W" becomes “74,1,44.23”

Wouldn’t the simple binary version be better, that is, “-74.0289527777778”?

I’d argue for this because, first, the current value has no indication of west or east, even though that information is in the original exif tag. (I don’t have any images from south of the equator, but I assume the same thing is happening for south.) I could grab the longitude reference tag, which is either W or E, but that would make it a separate field. Second I’m not familiar with the current version being used elsewhere.

OTOH, many systems using northing and easting, which is what the binary version of the exif tag is, I think. This includes GeoJSON.

I did try to play with the datatype in the template, but Tropy kept showing the same value no matter what I picked (which seems odd).

PS I’ve got a GitHub issue on how Tropy handles some other binary exif tag values.

I need to check, but if I remember correctly Exif actually stores the full information across several tags (or that there are extra tags which guide the interpretation of the lat/long tags). We would like to introduce more complex value data types in the future, which would allow for specialized widgets (e.g., a combined view of the coordinates or even the possibility to view/move a point on a map). This also covers other binary values you mentioned: for flash, for example, I think we’d want the widget to be a select field for the standardized values, using translatable labels to display the values etc.

In any case, these are good points! I’ll post more over on GitHub.

Some learning later…exif gps data is definitely a little complicated and depending on the image file, it’s present in slightly different forms. exiftool will return the hemisphere info if it can, it seems, but sometimes Tropy won’t find the data even when it’s present. A file that has XMP-exif data for GPS and also composite data gets nothing loaded for the lat and long in a template that uses exif: GPSLatitude, whereas a file with GPS data loads just fine. An example of the former:

> exiftool -G1 -a -GPS* Downloads/IMG_6815.jpg
[XMP-exif]      GPS Altitude                    : 114.03 m
[XMP-exif]      GPS Altitude Ref                : Above Sea Level
[XMP-exif]      GPS Latitude                    : 37 deg 48' 32.30" N
[XMP-exif]      GPS Longitude                   : 122 deg 12' 23.23" W
[Composite]     GPS Altitude                    : 114 m Above Sea Level
[Composite]     GPS Latitude Ref                : North
[Composite]     GPS Longitude Ref               : West
[Composite]     GPS Position                    : 37 deg 48' 32.30" N, 122 deg 12' 23.23" W

Running the binary version yields

> exiftool -b -G1 -a -GPS* -overwrite_original_in_place Downloads/IMG_6815.jpg
114.03
0
37.80897222
-122.20645278
114.03
N
W
37.80897222 -122.20645278

That would seem to be a bug.

Could you share the full XMP content? I don’t know what exiftool means by XMP-exif exactly. If there are Exif RDF properties present in XMP body then they would actually clash with the actual Exif tags (if both are present in the file). I’m not sure which would win out during the import, but it’s certainly something that we did not anticipate – if you could share the full XMP body, I’d be curious to take a look.

Sure. I think it’s a cropped image I edited in the iOS Photos app.

> exiftool -G1 -a Downloads/IMG_6815.jpg

[ExifTool]      ExifTool Version Number         : 12.16
[System]        File Name                       : IMG_6815.jpg
[System]        Directory                       : Downloads
[System]        File Size                       : 845 KiB
[System]        File Modification Date/Time     : 2019:06:01 19:41:54-04:00
[System]        File Access Date/Time           : 2021:02:01 17:10:50-05:00
[System]        File Inode Change Date/Time     : 2021:02:01 17:10:49-05:00
[System]        File Permissions                : rw-r--r--
[File]          File Type                       : JPEG
[File]          File Type Extension             : jpg
[File]          MIME Type                       : image/jpeg
[File]          Current IPTC Digest             : 5a5b926d07851ca0e35e61f3bd2b8a5a
[File]          Image Width                     : 2137
[File]          Image Height                    : 3866
[File]          Encoding Process                : Progressive DCT, Huffman coding
[File]          Bits Per Sample                 : 8
[File]          Color Components                : 3
[File]          Y Cb Cr Sub Sampling            : YCbCr4:2:2 (2 1)
[JFIF]          JFIF Version                    : 1.01
[JFIF]          Resolution Unit                 : inches
[JFIF]          X Resolution                    : 72
[JFIF]          Y Resolution                    : 72
[XMP-x]         XMP Toolkit                     : XMP Core 5.6.0
[XMP-xmp]       Creator Tool                    : 12.3.1
[XMP-xmp]       Modify Date                     : 2019:06:01 16:41:53-07:00
[XMP-xmp]       Create Date                     : 2019:06:01 16:29:58
[XMP-photoshop] Date Created                    : 2019:06:01 16:29:58
[XMP-exif]      Exposure Time                   : 1/40
[XMP-exif]      F Number                        : 1.8
[XMP-exif]      Exposure Program                : Program AE
[XMP-exif]      Date/Time Original              : 2019:06:01 16:29:58
[XMP-exif]      Brightness Value                : 4.92376833714916
[XMP-exif]      Exposure Compensation           : 0
[XMP-exif]      Metering Mode                   : Multi-segment
[XMP-exif]      Focal Length                    : 4.0 mm
[XMP-exif]      Exif Image Width                : 2898
[XMP-exif]      Exif Image Height               : 3866
[XMP-exif]      Exposure Mode                   : Auto
[XMP-exif]      White Balance                   : Auto
[XMP-exif]      Focal Length In 35mm Format     : 28 mm
[XMP-exif]      Scene Capture Type              : Standard
[XMP-exif]      GPS Altitude                    : 114.03 m
[XMP-exif]      GPS Altitude Ref                : Above Sea Level
[XMP-exif]      GPS Latitude                    : 37 deg 48' 32.30" N
[XMP-exif]      GPS Longitude                   : 122 deg 12' 23.23" W
[ICC-header]    Profile CMM Type                : Apple Computer Inc.
[ICC-header]    Profile Version                 : 4.0.0
[ICC-header]    Profile Class                   : Display Device Profile
[ICC-header]    Color Space Data                : RGB
[ICC-header]    Profile Connection Space        : XYZ
[ICC-header]    Profile Date Time               : 2017:07:07 13:22:32
[ICC-header]    Profile File Signature          : acsp
[ICC-header]    Primary Platform                : Apple Computer Inc.
[ICC-header]    CMM Flags                       : Not Embedded, Independent
[ICC-header]    Device Manufacturer             : Apple Computer Inc.
[ICC-header]    Device Model                    : 
[ICC-header]    Device Attributes               : Reflective, Glossy, Positive, Color
[ICC-header]    Rendering Intent                : Perceptual
[ICC-header]    Connection Space Illuminant     : 0.9642 1 0.82491
[ICC-header]    Profile Creator                 : Apple Computer Inc.
[ICC-header]    Profile ID                      : ca1a9582257f104d389913d5d1ea1582
[ICC_Profile]   Profile Description             : Display P3
[ICC_Profile]   Profile Copyright               : Copyright Apple Inc., 2017
[ICC_Profile]   Media White Point               : 0.95045 1 1.08905
[ICC_Profile]   Red Matrix Column               : 0.51512 0.2412 -0.00105
[ICC_Profile]   Green Matrix Column             : 0.29198 0.69225 0.04189
[ICC_Profile]   Blue Matrix Column              : 0.1571 0.06657 0.78407
[ICC_Profile]   Red Tone Reproduction Curve     : (Binary data 32 bytes, use -b option to extract)
[ICC_Profile]   Chromatic Adaptation            : 1.04788 0.02292 -0.0502 0.02959 0.99048 -0.01706 -0.00923 0.01508 0.75168
[ICC_Profile]   Blue Tone Reproduction Curve    : (Binary data 32 bytes, use -b option to extract)
[ICC_Profile]   Green Tone Reproduction Curve   : (Binary data 32 bytes, use -b option to extract)
[IPTC]          Coded Character Set             : UTF8
[IPTC]          Application Record Version      : 2
[IPTC]          Date Created                    : 2019:06:01
[IPTC]          Digital Creation Date           : 2019:06:01
[IPTC]          Time Created                    : 16:29:58
[IPTC]          Digital Creation Time           : 16:29:58
[Composite]     Aperture                        : 1.8
[Composite]     Image Size                      : 2137x3866
[Composite]     Megapixels                      : 8.3
[Composite]     Scale Factor To 35 mm Equivalent: 7.0
[Composite]     Shutter Speed                   : 1/40
[Composite]     GPS Altitude                    : 114 m Above Sea Level
[Composite]     Date/Time Created               : 2019:06:01 16:29:58
[Composite]     Digital Creation Date/Time      : 2019:06:01 16:29:58
[Composite]     GPS Latitude Ref                : North
[Composite]     GPS Longitude Ref               : West
[Composite]     Circle Of Confusion             : 0.004 mm
[Composite]     Field Of View                   : 65.5 deg
[Composite]     Focal Length                    : 4.0 mm (35 mm equivalent: 28.0 mm)
[Composite]     GPS Position                    : 37 deg 48' 32.30" N, 122 deg 12' 23.23" W
[Composite]     Hyperfocal Distance             : 2.07 m

I would need the actual XMP (the XML embedded in the file, or the image itself if it’s OK to share it here). Thanks!

How’s this?

> exiftool -xmp:all -j Downloads/IMG_6815.jpg
[{
  "SourceFile": "Downloads/IMG_6815.jpg",
  "XMPToolkit": "XMP Core 5.6.0",
  "CreatorTool": "12.3.1",
  "ModifyDate": "2019:06:01 16:41:53-07:00",
  "CreateDate": "2019:06:01 16:29:58",
  "ExposureTime": "1/40",
  "FNumber": 1.8,
  "ExposureProgram": "Program AE",
  "DateTimeOriginal": "2019:06:01 16:29:58",
  "BrightnessValue": 4.92376833714916,
  "ExposureCompensation": 0,
  "MeteringMode": "Multi-segment",
  "FocalLength": "4.0 mm",
  "ExifImageWidth": 2898,
  "ExifImageHeight": 3866,
  "ExposureMode": "Auto",
  "WhiteBalance": "Auto",
  "FocalLengthIn35mmFormat": "28 mm",
  "SceneCaptureType": "Standard",
  "GPSAltitudeRef": "Above Sea Level",
  "GPSLatitude": "37 deg 48' 32.30\" N",
  "GPSLongitude": "122 deg 12' 23.23\" W"
}]

XML version:

<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 5.6.0">
   <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
      <rdf:Description rdf:about=""
            xmlns:xmp="http://ns.adobe.com/xap/1.0/"
            xmlns:photoshop="http://ns.adobe.com/photoshop/1.0/"
            xmlns:exif="http://ns.adobe.com/exif/1.0/">
         <xmp:CreatorTool>12.3.1</xmp:CreatorTool>
         <xmp:ModifyDate>2019-06-01T16:41:53-07:00</xmp:ModifyDate>
         <xmp:CreateDate>2019-06-01T16:29:58</xmp:CreateDate>
         <photoshop:DateCreated>2019-06-01T16:29:58</photoshop:DateCreated>
         <exif:ExposureTime>1/40</exif:ExposureTime>
         <exif:FNumber>9/5</exif:FNumber>
         <exif:ExposureProgram>2</exif:ExposureProgram>
         <exif:DateTimeOriginal>2019-06-01T16:29:58</exif:DateTimeOriginal>
         <exif:BrightnessValue>58066/11793</exif:BrightnessValue>
         <exif:ExposureBiasValue>0/1</exif:ExposureBiasValue>
         <exif:MeteringMode>5</exif:MeteringMode>
         <exif:FocalLength>399/100</exif:FocalLength>
         <exif:PixelXDimension>2898</exif:PixelXDimension>
         <exif:PixelYDimension>3866</exif:PixelYDimension>
         <exif:ExposureMode>0</exif:ExposureMode>
         <exif:WhiteBalance>0</exif:WhiteBalance>
         <exif:FocalLengthIn35mmFilm>28</exif:FocalLengthIn35mmFilm>
         <exif:SceneCaptureType>0</exif:SceneCaptureType>
         <exif:GPSAltitude>114.03</exif:GPSAltitude>
         <exif:GPSAltitudeRef>0</exif:GPSAltitudeRef>
         <exif:GPSLatitude>37.80897222N</exif:GPSLatitude>
         <exif:GPSLongitude>122.20645278W</exif:GPSLongitude>
      </rdf:Description>
   </rdf:RDF>
</x:xmpmeta>
<?xpacket end="w"?>

Thanks, this explains it. It looks like Adobe has introduced their own Exif namespace. Tropy uses the W3 namespace which has been around for much longer. That’s why those tags don’t get imported at the moment. You could import them by importing Adobe’s Exif vocabulary and using their properties in your template: then Tropy should pick them up, but of course it’s not ideal to have two vocabularies for the same thing.

We should research if there is some consensus as to which vocabulary to use. If Adobe’s is to be preferred, we can switch vocabularies in Tropy and convert existing data during an update.

Alternatively, if this namespace is used exclusively in XMP we could convert it during import.

Hmmm. That’s a little odd because I don’t use Adobe products, but I did use an online tool to get the xml version of the xmp. What tool do you use to get the xmp out? Exiftool gave me the json. Or I can just send you the image.

The XMP standard was originally created by Adobe, hence most of the vocabulary is in the Adobe namespace, so this XMP data looks totally fine and I’m sure it’s embedded this way in your file.

In any case, this explains why Tropy didn’t find some of the Exif data and that’s something we should address by deciding how to deal with the two different namespaces. Ideally, we should use one and Tropy should be able to convert the other one automatically.

The bigger issue here is complex/composite values. We need to improve Tropy’s capabilities to handle complex data types (beyond simple strings and numbers) before properly addressing that.

OK, I somehow missed the -X flag in exiftool. Here’s the output from that instead of the on-line tool. You’ll see that the Adobe stuff is gone.

> exiftool -X -xmp:all Downloads/IMG_6815.jpg
<?xml version='1.0' encoding='UTF-8'?>
<rdf:RDF xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#'>

<rdf:Description rdf:about='Downloads/IMG_6815.jpg'
  xmlns:et='http://ns.exiftool.ca/1.0/' et:toolkit='Image::ExifTool 12.16'
  xmlns:XMP-x='http://ns.exiftool.ca/XMP/XMP-x/1.0/'
  xmlns:XMP-xmp='http://ns.exiftool.ca/XMP/XMP-xmp/1.0/'
  xmlns:XMP-photoshop='http://ns.exiftool.ca/XMP/XMP-photoshop/1.0/'
  xmlns:XMP-exif='http://ns.exiftool.ca/XMP/XMP-exif/1.0/'>
 <XMP-x:XMPToolkit>XMP Core 5.6.0</XMP-x:XMPToolkit>
 <XMP-xmp:CreatorTool>12.3.1</XMP-xmp:CreatorTool>
 <XMP-xmp:ModifyDate>2019:06:01 16:41:53-07:00</XMP-xmp:ModifyDate>
 <XMP-xmp:CreateDate>2019:06:01 16:29:58</XMP-xmp:CreateDate>
 <XMP-photoshop:DateCreated>2019:06:01 16:29:58</XMP-photoshop:DateCreated>
 <XMP-exif:ExposureTime>1/40</XMP-exif:ExposureTime>
 <XMP-exif:FNumber>1.8</XMP-exif:FNumber>
 <XMP-exif:ExposureProgram>Program AE</XMP-exif:ExposureProgram>
 <XMP-exif:DateTimeOriginal>2019:06:01 16:29:58</XMP-exif:DateTimeOriginal>
 <XMP-exif:BrightnessValue>4.92376833714916</XMP-exif:BrightnessValue>
 <XMP-exif:ExposureCompensation>0</XMP-exif:ExposureCompensation>
 <XMP-exif:MeteringMode>Multi-segment</XMP-exif:MeteringMode>
 <XMP-exif:FocalLength>4.0 mm</XMP-exif:FocalLength>
 <XMP-exif:ExifImageWidth>2898</XMP-exif:ExifImageWidth>
 <XMP-exif:ExifImageHeight>3866</XMP-exif:ExifImageHeight>
 <XMP-exif:ExposureMode>Auto</XMP-exif:ExposureMode>
 <XMP-exif:WhiteBalance>Auto</XMP-exif:WhiteBalance>
 <XMP-exif:FocalLengthIn35mmFormat>28 mm</XMP-exif:FocalLengthIn35mmFormat>
 <XMP-exif:SceneCaptureType>Standard</XMP-exif:SceneCaptureType>
 <XMP-exif:GPSAltitude>114.03 m</XMP-exif:GPSAltitude>
 <XMP-exif:GPSAltitudeRef>Above Sea Level</XMP-exif:GPSAltitudeRef>
 <XMP-exif:GPSLatitude>37 deg 48&#39; 32.30&quot; N</XMP-exif:GPSLatitude>
 <XMP-exif:GPSLongitude>122 deg 12&#39; 23.23&quot; W</XMP-exif:GPSLongitude>
</rdf:Description>
</rdf:RDF>

OK now I would be curious to know what’s actually embedded in the file. If you could share the original file I’ll take a look. This one here is not better, because it’s just a custom namespace for ‘xmp-exif’ added by exiftool by the looks of it.

I was wondering if that wasn’t happening with both. Messaged you the file.

@inukshuk Any luck with that file?

Yes, the XMP data looks like what you extracted using the online service you mentioned. It’s using Adobe’s NS. We’ll have to survey best practices in this regard and then either drop the W3 namespace in favor of Adobe’s or convert Adobe’s namespace when import XMP data.

I’m not terribly well informed on this, but as long as my files get read correctly, I’m good. This is stopping me from importing files right now.

(That and the ability to reload exif data if I change templates. If this was possible, I could load the images with the expectation of getting unimported gps data in the future.)