Tom Mitchell

Building data warehouses for fintech & investment firms.
Writing about architecture, consulting & high-value data engineering.

[Tom Mitchell @tommitchelldata]

Today I start a new consulting role as a data engineer for a London based company.

I'll be using tools like SQL, Python, Airflow, DBT and GCP.

I'm thinking about documenting the role day-by-day, challenges, tasks, my thought process etc - would that be interesting to you?

[Tom Mitchell @tommitchelldata]

I hope you've found this thread helpful.

If you did, I ask for 2 small favours:

1. Follow me @imtommitchell for more like this daily.

2. Click below, jump to the top and share to help someone else.

[Tom Mitchell @tommitchelldata]

P.S. If you're interested in data you'll love my weekly newsletter.

I share everything I know about building high-paying data skills from my 8+ years in the industry.

Subscribe for free here:

https://t.co/sAbqnnIU2Z

[Tom Mitchell @tommitchelldata]

Knowing how to select data in SQL isn't enough.

You need to know how to join tables.

To join tables correctly you need to understand:

- Primary keys
- Foreign keys
- Unique keys

Here's my super-simple breakdown of each one:

[Tom Mitchell @tommitchelldata]

TL;DR:

- Primary Keys are unique tags for each row of data in a table like customer ID

- Foreign keys act as Part B in the link and can point to a Primary Key (part A) in another table.

- Unique Keys are a column (or columns) that contain unique values. Not the primary identifier.

Thanks for reading.

If you liked this, follow me @imtommitchell for more content like this every day.

[Tom Mitchell @tommitchelldata]

3. Unique Key

A unique key is similar to a primary key, but it doesn't have to be the main identifier for a table. It's a column (or columns) that contains unique values, like a primary key.

However, a table can have multiple unique keys, while it can only have one primary key.

Unique keys are handy when you want to make sure that certain columns have distinct values, but they don't serve as the primary way to identify each row.

For example, in a customer table, you might have a unique key for email addresses to prevent duplicate email entries.

[Tom Mitchell @tommitchelldata]

P.S. If you're interested in data you'll love my weekly newsletter.

I share everything I know about building high-paying data skills from my 8+ years in the industry.

Subscribe for free here:

https://t.co/sAbqnnIU2Z

[Tom Mitchell @tommitchelldata]

I never realised how much I would need debating skills working in data.

Data talks but that doesn't mean people listen.

To debate effectively you need to:

- Put yourself in your stakeholder's shoes.
- Anticipate pushbacks.
- Structure your argument around your claim.

[Tom Mitchell @tommitchelldata]

P.S. If you're interested in data you'll love my weekly newsletter.

I share everything I know about building high-paying data skills from my 8+ years in the industry.

Subscribe for free here:

https://t.co/sAbqnnIU2Z

[Tom Mitchell @tommitchelldata]

Don't aim for this:

Excel - 100%
SQL - 0%
PowerBI/Tableau - 0%
Python/R - 0%

Aim for this:

Excel - 25%
SQL - 25%
PowerBI/Tableau - 25%
Python/R - 25%

You don't need to know everything straight away.

[Tom Mitchell @tommitchelldata]

P.S. If you're interested in data you'll love my weekly newsletter.

I share everything I know about building high-paying data skills from my 8+ years in the industry.

Subscribe for free here:

https://t.co/sAbqnnIU2Z

[Tom Mitchell @tommitchelldata]

Data cleaning is one of the most important skills for a data analyst.

Not Excel.
Not SQL.
Not PowerBI

Without clean data, any analysis done is unreliable.

Here's my data cleaning 101:

[Tom Mitchell @tommitchelldata]

And there you have it - my data cleaning 101.

TL;DR:

- Handle duplicates and missing data
- Fix naming conventions in values and columns.
- Validate and test: does it make sense/look right?

Thank you for your time.

If you got something from this post, consider following me @imtommitchell

I post data-related content daily.

[Tom Mitchell @tommitchelldata]

Finally, validate and test.

This is the most crucial part.

Do some test aggregations and visualisations, and ask yourself:

- Does the data make sense?
- Does it prove or disprove your theory?
- Is there anything that doesn't "feel" right when observing?