Topics

#amazon #amazon-photos #api #archive #automation #data

Language breakdown

Python 99.7%

Shell 0.3%

▸ Files click to expand

README

Amazon Photos API

Installation

pip install amazon-photos -U

Output Examples

ap.db

| | dateTimeDigitized | id | name | ... | model | apertureValue | focalLength | width | height | size | |--:|:-------------------------|:-----------------------|:------------------|:----|:------------------|:--------------|:------------|------:|-------:|-------:| | 0 | 2019-07-06T18:22:00.000Z | HeMReF-vvJiTTkdPIeWuoP | 1694252973839.png | ... | iPhone XS | 54823/32325 | 17/4 | 3024 | 4032 | 432777 | | 1 | 2023-01-18T09:36:22.000Z | z_HiIvASAKqWmdrkjWiqMZ | 1692626817154.jpg | ... | iPhone XS | 54823/32325 | 17/4 | 3024 | 4032 | 234257 | | 2 | 2022-08-14T14:13:21.000Z | LKXEZbqoVrhrOYBezisGEQ | 1798219686789.jpg | ... | iPhone 11 Pro Max | 54823/32325 | 17/4 | 3024 | 4032 | 423987 | | 3 | 2020-06-28T19:32:30.000Z | EPUeciHtfKkGiYkfUyEuMa | 1593482220567.jpg | ... | iPhone XS | 54823/32325 | 17/4 | 3024 | 4032 | 898957 | | 4 | 2021-07-07T17:12:55.000Z | fdfKzRJbEyoVeGcfCoJgE- | 1592299282720.png | ... | iPhone XR | 54823/32325 | 17/4 | 3024 | 4032 | 432556 | | 5 | 2021-08-18T18:32:41.000Z | crskJSmKPFRhxbpfkivyLm | 1592902159105.png | ... | iPhone XR | 54823/32325 | 17/4 | 3024 | 4032 | 123123 | | 6 | 2023-08-23T19:12:21.000Z | qkBFUlyIdkUwVVSaVWWKEF | 1598138358650.png | ... | iPhone 11 | 54823/32325 | 17/4 | 3024 | 4032 | 437887 | | 7 | 2021-06-19T17:14:13.000Z | TXKMKC-mHvSUrtRfwmtyDe | 1622199863606.jpg | ... | iPhone 12 Pro | 14447/10653 | 21/5 | 1536 | 2048 | 758432 | | 8 | 2023-02-15T22:45:40.000Z | FRDvvjcZdpFWiwrIZfTNHO | 1581874518054.jpg | ... | iPhone 8 Plus | 54823/32325 | 399/100 | 1348 | 2049 | 862883 |

ap.print_tree()

~ 
├── Documents 
├── Pictures 
│   ├── iPhone 
│   └── Web 
│       ├── foo 
│       └── bar
├── Videos 
└── Backup 
    ├── LAPTOP-XYZ 
    │   └── Desktop 
    └── DESKTOP-IJK 
        └── Desktop

Setup

[Update] Jan 04 2024: To avoid confusion, setting env vars is no longer supported. One must pass cookies directly as

shown below.

Log in to Amazon Photos and copy the following cookies:

session-id
ubid*
at*

Canada/Europe

where xx is the TLD (top-level domain)

ubid-acbxx
at-acbxx

United States

ubid_main
at_main

E.g.

from amazon_photos import AmazonPhotos
ap = AmazonPhotos(     ## US     # cookies={     #     'ubid_main': ...,     #     'at_main': ...,     #     'session-id': ...,     # },
## Canada     # cookies={     #     'ubid-acbca': ...,     #     'at-acbca': ...,     #     'session-id': ...,     # }
## Italy     # cookies={     #     'ubid-acbit': ...,     #     'at-acbit': ...,     #     'session-id': ...,     # } )

Examples

A database named ap.parquet will be created during the initial setup. This is mainly used to reduce upload conflicts

by checking your local file(s) md5 against the database before sending the request.

from amazon_photos import AmazonPhotos
ap = AmazonPhotos(     # see cookie examples above     cookies={...},     # optionally cache all intermediate JSON responses     tmp='tmp',     # pandas options     dtype_backend='pyarrow',     engine='pyarrow', )
get current usage stats
ap.usage()
get entire Amazon Photos library
nodes = ap.query("type:(PHOTOS OR VIDEOS)")
query Amazon Photos library with more filters applied
nodes = ap.query("type:(PHOTOS OR VIDEOS) AND things:(plant AND beach OR moon) AND timeYear:(2023) AND timeMonth:(8) AND timeDay:(14) AND location:(CAN#BC#Vancouver)")
sample first 10 nodes
node_ids = nodes.id[:10]
move a batch of images/videos to the trash bin
ap.trash(node_ids)
get trash bin contents
ap.trashed()
permanently delete a batch of images/videos
ap.delete(node_ids)
restore a batch of images/videos from the trash bin
ap.restore(node_ids)
upload media (preserves local directory structure and copies to Amazon Photos root directory)
ap.upload('path/to/files')
download a batch of images/videos
ap.download(node_ids)
convenience method to get photos only
ap.photos()
convenience method to get videos only
ap.videos()
get all identifiers calculated by Amazon.
ap.aggregations(category="all")
get specific identifiers calculated by Amazon.
ap.aggregations(category="location")

Search

Undocumented API, current endpoints valid Dec 2023.

For valid location and people IDs, see the results from the aggregations() method.

| name | type | description | |:----------------|:-----|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | ContentType | str | "JSON" | | _ | int | 1690059771064 | | asset | str | "ALL"
"MOBILE"
"NONE
"DESKTOP"

default: "ALL" | | filters | str | "type:(PHOTOS OR VIDEOS) AND things:(plant AND beach OR moon) AND timeYear:(2019) AND timeMonth:(7) AND location:(CAN#BC#Vancouver) AND people:(CyChdySYdfj7DHsjdSHdy)"

default: "type:(PHOTOS OR VIDEOS)" | | groupByForTime | str | "day"
"month"
"year" | | limit | int | 200 | | lowResThumbnail | str | "true"
"false"

default: "true" | | resourceVersion | str | "V2" | | searchContext | str | "customer"
"all"
"unknown"
"family"
"groups"

default: "customer" | | sort | str | "['contentProperties.contentDate DESC']"
"['contentProperties.contentDate ASC']"
"['createdDate DESC']"
"['createdDate ASC']"
"['name DESC']"
"['name ASC']"

default: "['contentProperties.contentDate DESC']" | | tempLink | str | "false"
"true"

default: "false" | |

Nodes

Docs last updated in 2015

| FieldName | FieldType | Sort Allowed | Notes | |-------------------------------|--------------------------|--------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | isRoot | Boolean | | Only lower case "true" is supported. | | name | String | Yes | This field does an exact match on the name and prefix query. Consider node1{ "name" : "sample" } node2 { "name" : "sample1" } Query filter
name:sample will return node1
name:sample* will return node1 and node2 | | kind | String | Yes | To search for all the nodes which contains kind as FILE kind:FILE | | modifiedDate | Date (in ISO8601 Format) | Yes | To Search for all the nodes which has modified from time modifiedDate:{"2014-12-31T23:59:59.000Z" TO *] | | createdDate | Date (in ISO8601 Format) | Yes | To Search for all the nodes created on createdDate:2014-12-31T23:59:59.000Z | | labels | String Array | | Only Equality can be tested with arrays.
if labels contains ["name", "test", "sample"].
Label can be searched for name or combination of values.
To get all the labels which contain name and test
labels: (name AND test) | | description | String | | To Search all the nodes for description with value 'test'
description:test | | parents | String Array | | Only Equality can be tested with arrays.
if parents contains ["id1", "id2", "id3"].
Parent can be searched for name or combination of values.
To get all the parents which contains id1 and id2
parents:id1 AND parents:id2 | | status | String | Yes | For searching nodes with AVAILABLE status.
status:AVAILABLE | | contentProperties.size | Long | Yes | | | contentProperties.contentType | String | Yes | If prefix query, only the major content-type (e.g. image, video, etc.) is supported as a prefix. | | contentProperties.md5 | String | | | | contentProperties.contentDate | Date (in ISO8601 Format) | Yes | RangeQueries and equals queries can be used with this field | | contentProperties.extension | String | Yes | |

Restrictions

Max # of Filter Parameters Allowed is 8

| Filter Type | Filters | |:------------|:--------------------------------------------------------------------------------------| | Equality | createdDate, description, isRoot, kind, labels, modifiedDate, name, parentIds, status | | Range | contentProperties.contentDate, createdDate, modifiedDate | | Prefix | contentProperties.contentType, name |

Range Queries

| Operation | Syntax | |----------------------|------------------------------------------------------------------| | GreaterThan | {"valueToBeTested" TO *} | | GreaterThan or Equal | ["ValueToBeTested" TO *] | | LessThan | {* TO "ValueToBeTested"} | | LessThan or Equal | {* TO "ValueToBeTested"] | | Between | ["ValueToBeTestedLowerBound" TO "ValueToBeTestedUpperBound"] |

Notes

`https://www.amazon.ca/drive/v1/batchLink`

This endpoint is called when downloading a batch of photos/videos in the web interface. It then returns a URL to

download a zip file, then makes a request to that url to download the content. When making a request to download data for 1200 nodes (max batch size), it turns out to be much slower (~2.5 minutes) than asynchronously downloading 1200 photos/videos individually (~1 minute).

Known File Types

| Extension | Category | |-----------|----------| | \.pdf | pdf | | \.doc | doc | | \.docx | doc | | \.docm | doc | | \.dot | doc | | \.dotx | doc | | \.dotm | doc | | \.asd | doc | | \.cnv | doc | | \.mp3 | mp3 | | \.m4a | mp3 | | \.m4b | mp3 | | \.m4p | mp3 | | \.wav | mp3 | | \.aac | mp3 | | \.aif | mp3 | | \.mpa | mp3 | | \.wma | mp3 | | \.flac | mp3 | | \.mid | mp3 | | \.ogg | mp3 | | \.xls | xls | | \.xlm | xls | | \.xll | xls | | \.xlc | xls | | \.xar | xls | | \.xla | xls | | \.xlb | xls | | \.xlsb | xls | | \.xlsm | xls | | \.xlsx | xls | | \.xlt | xls | | \.xltm | xls | | \.xltx | xls | | \.xlw | xls | | \.ppt | ppt | | \.pptx | ppt | | \.ppa | ppt | | \.ppam | ppt | | \.pptm | ppt | | \.pps | ppt | | \.ppsm | ppt | | \.ppsx | ppt | | \.pot | ppt | | \.potm | ppt | | \.potx | ppt | | \.sldm | ppt | | \.sldx | ppt | | \.txt | txt | | \.text | txt | | \.rtf | txt | | \.xml | markup | | \.htm | markup | | \.html | markup | | \.zip | zip | | \.rar | zip | | \.7z | zip | | \.jpg | img | | \.jpeg | img | | \.png | img | | \.bmp | img | | \.gif | img | | \.tif | img | | \.svg | img | | \.mp4 | vid | | \.m4v | vid | | \.qt | vid | | \.mov | vid | | \.mpg | vid | | \.mpeg | vid | | \.3g2 | vid | | \.3gp | vid | | \.flv | vid | | \.f4v | vid | | \.asf | vid | | \.avi | vid | | \.wmv | vid | | \.swf | exe | | \.exe | exe | | \.dll | exe | | \.ax | exe | | \.ocx | exe | | \.rpm | exe |

Custom Image Labeling (Optional)

Categorize your images into folders using computer vision models.

pip install amazon-photos[extras] -U

See the Model List for a list of all available models.

Sample Models

Very Large

eva02basepatch14448.mimin22kftin22k_in1k

Large

eva02largepatch14448.mimm38mftin22k_in1k

Medium

eva02smallpatch14336.mimin22kftin1k
vitbasepatch16clip384.laion2bftin12k_in1k
vitbasepatch16clip384.openaiftin12k_in1k
caformerm36.sailin22kftin1k_384

Small

eva02tinypatch14336.mimin22kftin1k
tinyvit5m224.distin22kftin1k
edgenextsmall.usiin1k
xcittiny12p8384.fbdistin1k

run(
    'eva02basepatch14448.mimin22kftin22k_in1k',
    path_in='images',
    path_out='labeled',
    thresh=0.0,  # threshold for predictions, 0.9 means you want very confident predictions only
    topk=5,
    # window of predictions to check if using exclude or restrict, if set to 1, only the top prediction will be checked
    exclude=lambda x: re.search('boat|ocean', x, flags=re.I),
    # function to exclude classification of these predicted labels
    restrict=lambda x: re.search('sand|beach|sunset', x, flags=re.I),
    # function to restrict classification to only these predicted labels
    dataloader_options={
        'batch_size': 4,  #  adjust 
        'shuffle': False,
        'numworkers': psutil.cpucount(logical=False),  #  adjust 
        'pin_memory': True,
    },
    accumulate=False,
    # accumulate results in pathout, if False, everything in pathout will be deleted before running again
    device='cuda',
    naming_style='name',  # use human-readable label names, optionally use the label index or synset
    debug=0,
)

🔗 More in this category