Topics

#conversation #imessage #pandas #phone-number #python #texting

Language breakdown

No language data available.

▸ Files click to expand

README

iMessage Conversation Analyzer

iMessage Conversation Analyzer (ICA) is a fully-typed Python library (and CLI utility) that will read the contents of an iMessage conversation via the Messages database on macOS. You can then gather various metrics of interest from the messages in that conversation.

Much of this program was inspired by and built using findings from [this blog post by Yorgos Askalidis][blog-post].

[blog-post]: https://medium.com/@yaskalidis/heres-how-you-can-access-your-entire-imessage-history-on-your-mac-f8878276c6e9

Installation

Open a Terminal and run the following:

pip3 install imessage-conversation-analyzer

You can also install ICA via [uv][uv]:

uv tool install imessage-conversation-analyzer

[uv]: https://docs.astral.sh/uv/

Usage

The package includes both a Command Line API for simplicity/convenience, as well as a Python API for developers who want maximum flexibility.

Command Line API

To use ICA from the command line, run the ica command from the Terminal. The minimum required arguments are:

A path to an analyzer file to run, or the name of a built-in analyzer
The first and last name of the contact(s), via the --contact / -c flag

1. If the contact has no last name on record, you can just pass the first name 2. You can also pass any phone number or email address associated with the contact; keep in mind that analysis will still run on all phone numbers / email addresses associated with the contact, not just the one you specify 3. For group chats, simply pass multiple --contact / -c flags

Example

ica message_totals -c 'Thomas Riverstone' -c 'Daniel Brightingale'

The following outputs a table like:

Metric               Total
Messages             20036
Messages From Me      7000
Messages From Daniel  6501
Messages From Thomas  6535
Reactions             4880
Reactions From Me     1700
Reactions From Daniel 1675
Reactions From Thomas 1505
Days Messaged          115
Days Missed              0
Days With No Reply       0

Built-in analyzers

ICA includes several built-in analyzers out of the box:

message_totals: a summary of message and reaction counts, by person and in

total, as well as other insightful metrics

attachment_totals: lists count data by attachment type, including

number of Spotify links shared, YouTube videos, Apple Music, etc.

mostfrequentemojis: count data for the top 10 most frequently used emojis

across the entire conversation

totalsbyday: a comprehensive breakdown of message totals for every day

you and the other participants have been messaging in the conversation

transcript: a full, unedited transcript of every message, including

reactions, between you and the other participants (attachment files not included)

count_phrases: count the number of case-insensitive occurrences of any

arbitrary strings across all messages in a conversation (excluding reactions); use the -s / --case-sensitive option for case-sensitive counts, and the -r / --use-regex option to enable regular expression mode for all phrases you specify

from_sql: execute an arbitrary SQL query against the conversation data

(messages and attachments), using an in-memory SQLite database

Filtering

There are several built-in flags you can use to filter messages and attachments.

--from-date: A start date to filter messages by (inclusive); the format must

be ISO 8601-compliant, e.g. YYYY-MM-DD or YYYY-MM-DDTHH:MM:SS

--to-date: An end date to filter messages by (exclusive); the format must be

ISO 8601-compliant, e.g. YYYY-MM-DD or YYYY-MM-DDTHH:MM:SS

--from-person / -p: A reference to the person by whom to filter messages;

accepted values can be me, them, all, or another participant; you can specifying another participant using their first name, full name, phone number, or email address (defaults to all); to filter by multiple people, pass this flag multiple times (e.g. -p Thomas -p Daniel)

ica message_totals -c 'Thomas Riverstone' --from-date 2024-12-01 --to-date 2025-01-01 --from-person 'Thomas'

# Filtering by more than one person
ica message_totals -c 'Thomas Riverstone' -c 'Daniel Brightingale' --from-date 2024-12-01 --to-date 2025-01-01 --from-person 'Thomas' --from-person 'Jane'

Other formats

You can optionally pass the -f/--format flag to output to a specific format like CSV (supported formats include csv, excel/xlsx, markdown/md, and json).

ica message_totals -c 'Thomas Riverstone' -f csv

ica ./mycustomanalyzer.py -c 'Thomas Riverstone' -f csv

Writing to a file

Finally, there is an optional -o/--output flag if you want to output to a specified file. ICA will do its best to infer the format from the file extension, although you could also pass --format if you have special filename requirements.

ica transcript -c 'Thomas Riverstone' -o ./my_transcript.xlsx

Python API

The Python API is much more powerful, allowing you to integrate ICA into any type of Python project that can run on macOS. All of the built-in analyzers (under the ica/analyzers directory) actually use this API.

Here's a complete example that shows how to retrieve the transcript of an entire iMessage conversation with one or more other people.

# getmytranscript.py
import pandas as pd
import ica
Export a transcript of the entire conversation
def main() -> None:
    # Allow your program to accept all the same CLI arguments as the ica
    # command; you can skip calling this if have other means of specifying the
    # contact name and output format; you can also add your own arguments this
    # way (see the count_phrases analyzer for an example of this)
    cliargs = ica.getcliparser().parseargs(
        namespace=ica.TypedCLIArguments()
    )
    # Retrieve the dataframes corresponding to the processed contents of the
    # database; dataframes include messages and attachments
    dfs = ica.get_dataframes(
        contacts=cli_args.contacts,
        timezone=cli_args.timezone,
        fromdate=cliargs.from_date,
        todate=cliargs.to_date,
        frompeople=cliargs.from_people,
    )
    # Send the results to stdout (or to file) in the given format
    ica.output_results(
        pd.DataFrame(
            {
                "timestamp": dfs.messages["datetime"],
                "isfromme": dfs.messages["isfromme"],
                "isreaction": dfs.messages["isreaction"],
                # U+FFFC is the object replacement character, which appears as
                # the textual message for every attachment
                "message": dfs.messages["text"].replace(
                    r"\ufffc", "(attachment)", regex=True
                ),
            }
        ),
        # The default format (None) corresponds to the pandas default dataframe
        # table format
        format=cli_args.format,
        # When output is None (the default), ICA will print to stdout
        output=cli_args.output,
        # Make certain column labels more human-friendly with
        # prettifiedlabeloverrides
        prettifiedlabeloverrides={
            'isfromme': 'Is from Me?',
            'is_reaction': 'Is Reaction?'
        }
    )
if name == "main":     main()

You can run the above program using the ica command, or execute it directly like any other Python program.

ica ./getmytranscript.py -c 'Thomas Riverstone'

python ./getmytranscript.py -c 'Thomas Riverstone'

python -m getmytranscript -c 'Thomas Riverstone'

You're not limited to writing a command line program, though! The ica.get_dataframes() function is the only function you will need in any analyzer program. But beyond that, feel free to import other modules, send your results to other processes, or whatever you need to do!

Errors and exceptions

BaseAnalyzerException: the base exception class for all library-related

errors and exceptions

ContactNotFoundError: raised if the specified contact was not found
ConversationNotFoundError: raised if the specified conversation was not

found

FormatNotSupportedError: raised if the specified format is not supported by

the library

Using a specific timezone

By default, all dates and times are in the local timezone of the system on which ICA is run. If you'd like to change this, you can pass the --timezone / -t option to the CLI with an [IANA timezone name][iana].

ica totalsbyday -c 'Daniel Brightingale' -t UTC

ica totalsbyday -c 'Daniel Brightingale' -t America/New_York

[iana]: https://data.iana.org/time-zones/tzdb-2021a/zone1970.tab

The equivalent option for the Python API is the timezone parameter to ica.get_dataframes:

dfs = ica.getdataframes(contact=mycontact, timez)

Data Schema

All analyzers (including the built-in from_sql analyzer and any custom analyzers you write) have access to the following dataframes/tables. An object with these dataframes are returned by the ica.get_dataframes() function in the Python API.

`messages`

A list of all messages in the conversation, including text messages and reactions.

`attachments`

A list of all attachments in the conversation, including images, videos, audio\, and any other types of files. Please note that no content is included, only metadata.

`handles`

A list of all handles (phone numbers and email addresses) associated with the participants of the conversation (other than the host user / "me"). This allows for easy joining with the messages dataframe.

SQL Functions

The Python API also exposes several powerful functions that allow you to query your conversation data using SQL. This is powered by an in-memory SQLite database that is automatically populated with the available iMessage dataframes. Please refer to the Data Schema section above for details on the available tables and their columns.

getsqlconnection(dfs): A context manager which creates a temporary in-memory SQLite database from your ICA dataframes, allowing you to operate on them with the ica.executesqlquery() function (documented below)
executesqlquery(query, con): Executes a SQL query against the connection provided by getsqlconnection; returns a pandas dataframe with the results

import ica
def main() -> None:     # Retrieve conversation data     dfs = ica.get_dataframes(contacts=["Jane Doe"])
# Run SQL queries against the data     with ica.getsqlconnection(dfs) as con:         results = ica.executesqlquery(             "SELECT * FROM messages WHERE isfromme = 1",             con         )         ica.output_results(results)
if name == "main":     main()

Developer Setup

The following instructions are written for developers who want to run the package locally or write their own analyzers.

We recommend using the uv package manager for easier environment and dependency management ([instructions][installation-docs]).

[installation-docs]: https://docs.astral.sh/uv/getting-started/installation/#installation-methods

1. Install uv

curl -LsSf https://astral.sh/uv/install.sh | sh

2. Create virtual environment and install dependencies

uv sync

3. Run CLI like normal

When you install ICA with uv, an editable installation of the package gets installed into the virtual environment that uv creates for you. This allows you to make changes to the source code and continue to invoke ica like normal:

ica message_totals -c 'Thomas Riverstone'

🔗 More in this category