Martin Schoombee

I help customers make sense of their data, using Microsoft technologies.

[Martin Schoombee @MartinSchoombee]

@SQLGene @mim_djo

[Martin Schoombee @MartinSchoombee]

@mim_djo @SQLGene It's like saying you should drive a BMW instead of Mercedes. Why? What's the reason apart from personal preference? And saying "just because it's better" is not enough in my opinion.

[Martin Schoombee @MartinSchoombee]

@mim_djo @SQLGene Of course...I don't think that's the point Eugene is asking about though. As consultants we're trying to provide the best guidance for our customers, and "do it because MS says so" is not great guidance without proof.

[Martin Schoombee @MartinSchoombee]

@SQLGene @mim_djo Yeah, I think the best we can do is: It'll be a game changer if they get it right, but until then it's better to use import mode.

[Martin Schoombee @MartinSchoombee]

@SQLGene @mim_djo Open source but not open source, welcome to the new world Eugene. Matter of time before somebody reverse engineers the algorithm though...I think in this case it's all about the Vertipaq engine and how to make it perform outside of using its own proprietary file format.

[Martin Schoombee @MartinSchoombee]

@mim_djo @SQLGene Haha, I think you know what Eugene means...he wants to know the internals, not just how to apply it.

[Martin Schoombee @MartinSchoombee]

@SQLGene @mim_djo Agreed, but I don't think they can do that without giving away some secrets. They ultimately want to give people a reason to use Fabric over any other Data Lake platform, and "Direct Lake" would be one of the few reasons.

[Martin Schoombee @MartinSchoombee]

@SQLGene @mim_djo I think this is fundamentally the reason for v-order. They couldn't get the Vertipaq engine to perform adequately on Delta Lake with the standard compression & sorting.

[Martin Schoombee @MartinSchoombee]

@MartyCatherall Thank you sir!

[Martin Schoombee @MartinSchoombee]

@SQLGene @mim_djo And/or they were able to adapt the Vertipaq engine to read Delta Lake files with some adjustments.

2/2

[Martin Schoombee @MartinSchoombee]

@SQLGene @mim_djo My guess, and a very uneducated one for that matter, is that the MS claim that suggests they can get similar performance from direct lake means:

By using their own sorting and compression (v-order), they get very close to the internal Vertipaq file format.

1/2

[Martin Schoombee @MartinSchoombee]

@FerrariAlberto Gets me every time :-)

[Martin Schoombee @MartinSchoombee]

@bob_duffy Hard "no" for me :-/

[Martin Schoombee @MartinSchoombee]

@erikdarlingdata Cupid...you don't have to worry about him this month.

[Carol Smit @CarolSmit17]

Do it for your country 🇿🇦 #proudlysouthafrican

How to Braai: Starting the Fire https://t.co/CczPA0BUNa
A feast of culture, braai and fine wine
https://t.co/hk5NqPqevk
The evolution of SA’s Heritage Day https://t.co/6nunE7ngyf

#desmondtutu #southafrica #ubuntu #28twelve

[Martin Schoombee @MartinSchoombee]

@GFritchey Describe "proper coffee" 😋

[Carol Smit @CarolSmit17]

#28twelve #powerbi #analytics in #southafrica

[Carol Smit @CarolSmit17]

There are some valuable lessons here for #data analysts and #businessintelligence professionals.

#analytics #cloud #digitaltransformation #28twelve

[Martin Schoombee @MartinSchoombee]

📝Blog Post: Rate my dashboard
https://t.co/0SJ5pBVGSI

[Martin Schoombee @MartinSchoombee]

@AndyLeonard Meant to say for Basket A items it's probably between 10% and 15%.

[Martin Schoombee @MartinSchoombee]

@AndyLeonard All different, depending on the market basket. I think for the top 100 items people buy (usually Basket A) it's probably that, but others it could be as high as 100%.