James Anderson

Husband, father, product manager and cat owner.

[James Anderson @DatabaseAvenger]

@redgate Orgs using fewer DB engines makes sense. Supporting one as a vendor is hard when customers need your product to work on multiple versions and cloud flavours which have many differences. ( (SQL Server 2017, 19, 22) * (Azure SQLDB, MI, Fabric, RDS, Cloud SQL and on-prem) ) = Lots!

[James Anderson @DatabaseAvenger]

Love DB migration failures https://t.co/8aTyc55fy9

[James Anderson @DatabaseAvenger]

@MrSiWhiteley why????

[James Anderson @DatabaseAvenger]

@philosophicles @davebally The best app is no app

[James Anderson @DatabaseAvenger]

@davebally That sounds ok as long as you can stub the response of the code calling the SP so that you don't require a SQL instance when testing. If not, testing will likely be slow and complicated. Integration testing will be important here too

[James Anderson @DatabaseAvenger]

@carlvellotti 🤖

[James Anderson @DatabaseAvenger]

@matsonj Just started using dbt_artifacts, but this looks further along.

[James Anderson @DatabaseAvenger]

@EdDebug @ChristianNolan A dbt project is just sql + jinja + yaml, but the app you run it with has so many things that you would have to code your self: logging, concurrency, config parsing, multiple table materialisation strategies, sql script dependency graph, validation, etc

[James Anderson @DatabaseAvenger]

@teej_m testing transformations is my biggest problem with dbt. This looks great. I basically want dbt to become pandas on rails

[James Anderson @DatabaseAvenger]

@bernhardsson CDC is OK, but an event can hit multiple tables meaning the event has to be inferred later in the pipeline by piecing all the context back together. If the app can produce an append only log of complete events it makes things much easier, faster and reliable

[James Anderson @DatabaseAvenger]

@BrentO And that's where I come in $$$

[Matthew Prince 🌥 @eastdakota]

Used to respect @hashicorp. Now… don’t. #staytuned

[James Anderson @DatabaseAvenger]

@kaxil @ApacheAirflow Dynamic tasks could be a game changer

[Kaxil Naik @kaxil]

🧵 Apache Airflow 2.3.0 is out! Soo many things to talk about 👇👇👇

➡️ This is the biggest @ApacheAirflow release since 2.0.0
➡️ 700+ commits since 2.2 including 50 new features, 99 improvements, 85 bug fixes

The following are the biggest & noteworthy changes👇👇👇:

[James Anderson @DatabaseAvenger]

@pedram_navid Recovering from few production outages can help build this confidence quickly

[James Anderson @DatabaseAvenger]

@bernhardsson For me SQL wins at most levels except when it comes to testing. Dataframe based code is far easier to test. Only important if you test your pipelines through 😄

[James Anderson @DatabaseAvenger]

@EdDebug @NowinskiK @AzureCosmosDB It's amazing so many of them had the same issue. Shell put up a temporary form late in the day. Bin the website and keep the async form

[James Anderson @DatabaseAvenger]

@dbafromthecold Nice. I'll be using some of this.