OnDataEngineering

Independent, critical and technical thinking on the use cases, architectural patterns and technologies relating to the preparation of data for exploitation.

[OnDataEngineering @OnDataEng]

Cloudera have announced the release of #ClouderaStreamsManagement - bundling their Kafka management console and replication tool #odenews https://t.co/dTon4OP3FN

[OnDataEngineering @OnDataEng]

Interested in #ApacheSamza, hear about LinkedIn's journey with it #odenews https://t.co/zTyFIljLNy

[OnDataEngineering @OnDataEng]

From the ever reliable The Morning Papers - Procella, YouTube's unified OLTP/OLAP (HTAP) database #odenews https://t.co/AGGE2ED5Xm

[OnDataEngineering @OnDataEng]

#InfluxDBCloud (their time series database as a service on AWS, Azure and GCP) has hit 2.0 and has now gone serverless #odenews https://t.co/cm1oNc1n6I

[OnDataEngineering @OnDataEng]

More on on #GoogleCloudDataproc Spark on GKE from ZDNet #odenews https://t.co/0KovlE5Qew

[OnDataEngineering @OnDataEng]

#GoogleCloudDataproc now supports (in alpha) running your Spark jobs on Kubernetes (GKE) rather than YARN VMs #odenews https://t.co/f7alYuUD4C

[OnDataEngineering @OnDataEng]

#QuboleDataService is now available on #GoogleCloud if you're looking for a cloud agnostic Hadoop as a service offering (that runs on Google Cloud) #odenews https://t.co/DHhVUICNqZ

[OnDataEngineering @OnDataEng]

Interesting in replicating data between #Kafka clusters - Cloudera have a post on on MirrorMaker 2 which is based on #KafkaConnect #odenews https://t.co/0t9UjsQhEk

[OnDataEngineering @OnDataEng]

#ApacheCalcite 1.21 is out - you might never have heard of it, but it's probably being used by many of the data tools you use on a daily basis for query parsing and optimization #odenews https://t.co/d6uny7WAnp

[OnDataEngineering @OnDataEng]

#ConfluentSchemaRegistry is now available as a cloud service in #ConfluentCloud #odenews https://t.co/w1WWxJfVfa

[OnDataEngineering @OnDataEng]

Looking for an open source object store - Datanami have the latest on #MinIO #odenews https://t.co/KVTFEgh7CS

[OnDataEngineering @OnDataEng]

And Amazon have also announced #AmazonEMR 6.0, with support for Hadoop 3.1 and running Spark jobs in Docker containers #odenews https://t.co/acYUdFLPoA

[OnDataEngineering @OnDataEng]

#AmazonEMR release 5.26 is out, with even better Spark performance #odenews https://t.co/mUHfZ1cMiQ

[OnDataEngineering @OnDataEng]

From Datanami, #Cloudera's Q2 results are better than expected #odenews https://t.co/xvXWRmnUj4

[OnDataEngineering @OnDataEng]

StreamSets have announced #StreamsetsTranformer - a graphical tool for creating #ApacheSpark pipelines that's part of their DataOps Platform #odenews @streamsets https://t.co/K8sfUhlksy

[OnDataEngineering @OnDataEng]

From Pivotal, their summary of what's great and good in the new #Greenplum 6.0 release #odenews https://t.co/CT8mn2oVHB

[OnDataEngineering @OnDataEng]

Looking for an open source object storage gateway that supports the Amazon S3 protocol - #ZenkoCloudserver has just released version 8.2 #odenews @zenko https://t.co/YB2Hlxd7HP

[OnDataEngineering @OnDataEng]

#ElasticCloud is now available on #Azure #odenews https://t.co/mQGfazn4Pv

[OnDataEngineering @OnDataEng]

Using #GoogleCloudStorage with #Hadoop - Google have a new version of their Cloud Storage Connector for Hadoop out with a bunch of performance improvements and locking for directory modifications #odenews https://t.co/dybFTpk0WE

[OnDataEngineering @OnDataEng]

From the ever excellent The Morning Paper, a review of a paper that used "the TPC-H benchmark to assess Redshift, Redshift Spectrum, Athena, Presto, Hive, and Vertica to find out what works best and the trade-offs involved" #odenews https://t.co/Zl8ajREP6r