Cascading

Need support or guidance?

Reach out to @cwensel

[Cascading @cascading]

It looks like the https://t.co/LHWS1eZnFo Maven repo was shutdown by its owners.

If you are having build failures, DM @cwensel for options to get back up.

[Cascading @cascading]

Cascading 4.5.2 is out!

https://t.co/NpUY30Plyp

https://t.co/1f5iJ9WBHo

[Julian Hyde @julianhyde]

Do you miss meetups of the Big Data era, when you could learn about RAFT protocol or map-side join over pizza and beer? Us too! My friend @cwensel has launched SF Distributed Systems Meetup.

First meetup is next week, with talks by me and @conor_power23 https://t.co/MmPRkvtPq8

[Cascading @cascading]

If your company is still running Cascading+HadoopMR applications and are itching to drop MR for Spark, dm me.

Some companies have 100s of @Cascading apps in production and moving them off MR in a couple lines of code would be a huge cost savings.

Also true for @scalding.

[Joe Hellerstein @joe_hellerstein]

I always suspected that Parquet and Orc weren't very good, but TIL from @ViktorLeis that one can do much much better with BtrBlocks. https://t.co/GpuZvQ0jaw

[Chris K Wensel @[email protected] @cwensel]

Cascading 4.5.1 was just released to Maven Central.

https://t.co/lAWvrayV91

This release includes minor bug fixes and dependency updates.

https://t.co/ottloXkhwo

[Julian Hyde @julianhyde]

@fuyufjh @andy_pavlo @ApacheCalcite @luciddb @iskyzh @ApacheHive Before I worked at Hortonworks putting Calcite in Hive, I worked on Lingual, a SQL interface to @cascading. That was the first commercial use of Calcite (then called Optiq) and some much needed consulting $$

[Chris K Wensel @[email protected] @cwensel]

I'm hanging my shingle out again.

I have some new insights I want time to work out into a developer tool for cloud data engineers, but I want to bootstrap it with contract/consulting revenue.

This is how I started @Cascading in 2007.

https://t.co/NfE0NE7UMx

Please boost!

[P. Oscar Boykin @posco]

Scala folks: if there are open source GitHub Twitter libraries you like, you might want to make a clone/fork now. Seems like we are in a situation where anything can happen.

[Cascading @cascading]

#MyTwitterAnniversary

[Chris K Wensel @[email protected] @cwensel]

really interested in adding mergeable summary as a datatype to @cascading. think having daily/hourly aggregate tables, then daily/hourly performing a last N day/hour aggregation against the partials. cost efficient way to have rolling percentiles/aggregates

[Cascading @cascading]

Cascading 4.5 is out!
With native @ApacheParquet, @ApacheKafka, @neo4j, and @splunk support.
- https://t.co/5iEuto5GSU
- https://t.co/zUe3nqmHOl
- https://t.co/7jQ3hQJ2LV

[Chris K Wensel @[email protected] @cwensel]

cross your fingers, we might have a @cascading 4.1 release tonight, and a quick 4.5 release a day later w/ Hadoop 3 support. Both will include native @ApacheParquet support to replace the support dropped by Parquet. Plus some well worn enhanced Parquet support in 4.6

[Chris K Wensel @[email protected] @cwensel]

For fun I really want to fork the @cascading local mode planner to support RocksDB as the backend for joins instead of the memory only hashmaps. Local mode was never intended to scale in joins.

[Cascading @cascading]

Cascading 4.1 is out!
With native @ApacheParquet, @ApacheKafka, @neo4j, and @splunk support.
- https://t.co/ZzCbgXht7M
- https://t.co/B1oF3CI1hw

[Chris K Wensel @[email protected] @cwensel]

finally @cascading has Apache Hadoop 3 (and Tez 0.10) support thanks to the team at @Foursquare Labs! https://t.co/fbf2O23Zt8

[Julian Hyde @julianhyde]

Gradle makes me feel stupid

[Steve Loughran [email protected] @steveloughran]

@cwensel @PinterestEng Google gs connector does do aligned block reads...both that and this one work well for backwards seeks (footers, stripe footers -> stripes)
/2

[Chris K Wensel @[email protected] @cwensel]

Improving efficiency and reducing runtime using S3 read optimization by @PinterestEng https://t.co/IBGHYnYQo5 << do you have a link to the git repo?

[Cascading @cascading]

Next up will be adding Parquet into the mix.

[Cascading @cascading]

The cascading-local repo has been merged into Cascading 4.1 providing direct support for Kafka, Splunk, S3, and Neo4j https://t.co/WgzVDFT8pd