General Folders

General Folders is an identity- and policy-aware transport layer for B2B data, replacing hacky SFTP, email, and ad-hoc pipelines.

Backed by top innovators 📈

[General Folders @GeneralFolders]

🆕 New podcast episode is out today, covering some of the frequently asked questions about General Folders.

Thank you @JakeSearch and @MatchRelevant for having @djpardis on your podcast, we loved the episode! :)

https://t.co/1xlJXwNg6a

[Ryan Petersen @typesfast]

@JeffLonsdale @ArjunMahadevan A few years ago I needed to transfer data between hospitals both using Epic via hand carried CD ROM.

[General Folders @GeneralFolders]

Here is the Medium version.
https://t.co/YvuLvDZRgK

[General Folders @GeneralFolders]

Who's in a reading mood this Friday?

@djpardis wrote about the evolution of software engineering from FORTRAN to LLMs to better understand the AI coding milestones underway.

It includes an in-depth look at major AI coding milestones.

https://t.co/H5m7uWWfqV

[Jacob Matson @matsonj]

@kellabyte But also a great idea

[tobi lutke @tobi]

@kellabyte Totally agreed. Very good idea.

[Kelly Sommers @kellabyte]

Now that ELT is so popular instead of ETL, the industry could use a standard protocol to define how raw replication of data is defined.

CDC is too fractured.

We need a way to replicate to down stream systems that connects as easily and reliably as replication typically does.

[the Rich @Duderichy]

when I found out bank transactions are cleared by ftp-ing files

[General Folders @GeneralFolders]

What an honor to rep @TechstarsSDSU alongside Misti Cain, Cathy Pucher, and the founders of StoryTap and Sigen AI! 💚

[Pete Hunt 🚁 @floydophone]

Ok, this is pretty impressive https://t.co/NdfTJj0t4E

[Simon Späti 🏔️ @sspaeti]

I uploaded 25+ new data engineering notes in preparation for my next book chapter #DataAssetReusabilityPattern.

The notes go from Apache Arrow Flight Protocol to Slowly Changing Dimensions Type 2. Here are some highlights:

↠ Apache Arrow Flight for efficient data transfer
↠ Apache Iceberg and ACID transactions for reliable data lakes
↠ Data locality and parametric pipelines for optimized processing
↠ WASM and the Declarative Data Stack for modern architectures
↠ Microservices, Protobuf, and schema registries for robust data systems
↠ Time travel capabilities and SCD Type 2 implementations

And much more. If you like these things, have a look at the «Data Engineering Vault»—links to dedicated notes below (including 8+ notes related to the Finding Flow article).

Stay tuned for the next book chapter coming out soon.

[Jacob Matson @matsonj]

@GeneralFolders "its literally a filing cabinet, inc"

[General Folders @GeneralFolders]

@matsonj Brb, changing our name

[Theo - t3.gg @theo]

Building a Google Drive clone and it's pushing my disdain towards file systems to an all time high.

"Folders" are an arbitrary way to organize files. There are so many weird behaviors we just take for granted.

Why can't a file be in another file? Why can't a folder have two parents? Why can't I filter for files that exist in two different folders?

[Theo - t3.gg @theo]

If folders worked as tags and not as hard filters, the “abstraction” would make way more sense. When you put something in a folder, it can no longer be in any other folder.

There’s a failure in specificity and in locality. The “single location” nature makes them nearly entirely useless outside of software dev architecture and URLs.

If you get too specific with how you break down your folders, everything becomes impossible to find outside of multi-directory search

If you don’t get specific enough, you are basically just reinventing file extensions

WinFS should have been completed and it breaks my heart it never was released

[General Folders @GeneralFolders]

Thank you for supporting us this past year! ♥️

Wishing you all an excellent 2025! 🎇🥳

[General Folders @GeneralFolders]

This excellent post discusses data platform options when working with your customers and vendors. Thank you for the shoutout @criccomini!

And we're loving the preview image! 😍🗂

[Taylor A Murphy @tayloramurphy]

they won't tell you this in DE school but the easiest way to fix missing data is to drop and do a nightly backfill...

[General Folders @GeneralFolders]

@DZhang50 @DbrxMosaicAI We were able to get a few pictures with it too! :D

[David J Phillips @davj]

Founders underestimate the power of truly listening to users

Users are the ones who will determine your success

[General Folders @GeneralFolders]

This excellent post discusses data platform options when working with your customers and vendors. Thank you for the shoutout @criccomini!

And we're loving the preview image! 😍🗂

[Chris @criccomini]

Last post of the year! S3, @ApacheParquet, @ApacheIceberg, and @duckdb are a great way to get customers their data.

https://t.co/t2nyOeKrcq