What your iMessage data says about you

Nowadays we communicate almost exclusively through text messages. Even my 96 year old grandmother is emoji literate and knows the true meaning of 🍆 and 🔥.

What would an analytic dive into those messages look like? What would we learn about relationships?

We’ll find out and look at building an application around these message visualizations in this article!

Have you ever, tilted a phone away as you wrote a private text?

Would you let a stranger scroll through your conversations?

Its almost uncomfortable to think how revealing and intimate our message history can be. It is a direct record of our relationships with those we deem worthy of texting back.

iMessage dominates the online messaging space due to its locked in nature with Apple devices.

Any developer or heavy computer user who owns a MacBook and iPhone know the unbelievable convenience that iMessages auto syncing is.

Okay, cool — well enough speculation. What does the data say?

Locked down data

Its not a surprise that this data is not easy to access. iOS has messages completely locked down — and not accessible through their SDK in any way.

Also with the iPhones locked down environment, we don’t have many options for locating and analyzing the messages on the device.

If only, there was a way for us to realtime export these messages…

💡AH HA! - The desktop app!

Our backdoor in

After a bit of research through Apple documentation we locate out iMessage DB on our MacBook.

If we run this simple command in the t erminal— we get the path to the database.

echo /Users/$USER/Library/Messages/chat.db

Finding the path to our messages database

Exploring/understanding this mess

We open the database and still its kinda confusing where the good data is. We still need to reverse engineer these database fields — or dig through Apple documentation.

Lets SQL around for awhile with DB Browser for SQLite — a simple application for messing with SQLite DBs.

chat.db loaded into DB Browser for SQLite

DB Browser for SQLite

After a few hours, I pieced together enough working SQL and clues from Apple docs to extract the information needed.

This work gets wrapped up in a Python class so I can do some analysis in an Jupyter Notebook.

Heres the snippet to extract the data

Now were equip to do some real analysis with Python — and then move some of that work into useful visualizations.

🤷🏽‍ emojis are important too!

Even without including the text in our analysis we can do some emoji fingerprinting.

We can get a good idea of a relationship, just by showing the number of each emoji sent between myself and another person.

This could be even more descriptive if broken out into sent — received.

Here are three different emoji grids with friends of mine

Pattern finding

Great we have pretty pictures, and did some aggregation counts on emojis.

However, we want to make sure this stuff makes sense the real world and do some sanity checking.

Here we’ll do a deeper analysis and try to tie together a few points of data we extracted.

Below is a hour by day chart with a family member I live with. Also the top emojis are shown on the right.

Immediately we notice the darker squares around 9pm, and the high count of dog emojis… To me, this pattern couldn’t be more clear.

My dog Ness; has his last walk around 9–10PM. Often my family member walks him if I am unavailable — or far from home.

These texts are asking if Ness needs to be walked, and sometimes come with a dog emoji.

Great — things are lining up.

“Necesitas que yo camine con Ness🐕 esta noche?”

— family member

All of that for this guy

Ness being dramatic on FT

Lets do it again — make an Application

Now we have this cool tool but want to make it reusable and repeatable. We are going to use NodeJS for the SQLite connection, Express to serve and Electron to package it all up.

We need to:

1. Build a UI — lets get inspiration from the actual iMessage app!
2. Classify the DB connector script — lets use NodeJS
3. Build a small sever to connect UI and DB
4. Make a small Electron app wrapper

General App Layout

Building the App

We use the python script written earlier as a starting point on the Node version. We do this language conversion because we eventually want to package this whole thing up into a AppStore Mac app. Electron makes this process easier and we want to keep all of our code in the same language.

First, lets make the electron app shell — a main.js file that will open a window listening to localhost and the port our UI will run on in dev.

Next, we use express to set up a small server. This sever should return the html page of the application, and have two endpoints — one to get messages and one to get the last messages; the side bar in iMessage.

You’ll notice that this server is instantiating the DataGrabber class that we have not built yet, this is going to be the translated python class.

When we build it, there are some changes because now we use promises to make sure data is returned instead of relying on python’s synchronous execution flow.

These are essentially all of the pieces needed to build out the app, we still need a lot of CSS, JS and HTML to make the UI. This I’ll leave up to the reader if they are interested.

❓Questions to the readers

1. What kind of information do you think we can extract from our conversations?
2. How do we do this?
3. What is the danger or benefits of doing this?
4. Is this information something people want to know about themselves?

Thanks for getting through this adventure with me. Please show support and 👏 (clap) for this article.
Remember you can clap up to 50 times, and clapping for this article would be really helpful in the Medium algorithm. 
Any support is greatly appreciated ❤️

Also, please answer the questions and add to the discussion below