James Fancke

Accountant that likes tinkering with data.
Also likes running and making lasagne.

[James Fancke @jdfancke]

It's been fun squeezing this exam in on the weekend...😊
I'm happy to share that I've just passed the DP-600 exam and have achieved the "Microsoft Fabric Analytics Engineer Associate" cert.

[James Fancke @jdfancke]

@JohannesVink @DataMozart @MSPowerBI Yes it looks like it's around 68k EUR for an F64 in West Europe after the reservation discount. Some regions are more expensive than others, it's about 20% more in West Europe than East US2. https://t.co/pxUvrCFwgR

[James Fancke @jdfancke]

@CJMajka Build a proof of concept model/report retrieving directly from the data sources before you build any infrastructure for movement of data (eg loading data into a data warehouse).

[James Fancke @jdfancke]

@eric_beery11 I wish it supported github flavoured markdown and diagramming as code packages like mermaid.js

[James Fancke @jdfancke]

@s_debruyn Great post! I just stumbled across your blog for the first time and thought I'd let you know the content you put out is great 😊

[James Fancke @jdfancke]

@GregBaldini @mim_djo @PawarBI Also somewhat interesting, the endpoint for staging was in a different region (my "local" region WEU) vs the capacity region (EUS2), plus the endpoints did not appear to be those described by the docs. I can imagine getting corporate firewalls opened for 1433 will be a challenge.

[James Fancke @jdfancke]

Finally had some time to start testing Fabric 😀
Having trouble with loading a DWH using DfGen2 (probably due to firewall rules). Below is my very rough (and probably incorrect understanding)... Any experts out there already been through this early hurdle? @mim_djo @PawarBI?

[James Fancke @jdfancke]

@GregBaldini @mim_djo @PawarBI Did some brief tests, it looks like indeed it is doing a round trip.
Staging enabled = Gateway to Staging via 443, Staging back to Gateway via 443, Local to Destination via 1433.
Staging disabled = Gateway to Destination via 1433.
Gateway only queries source once in either case.

[James Fancke @jdfancke]

@marcorus Spaces because I used to edit . bim files manually in a text editor and hated when you'd see a \t

[James Fancke @jdfancke]

@GregBaldini @mim_djo @PawarBI I think from my original image I just need to delete the 3rd line (nothing from staging goes back to the gateway)... I hope. But it does seem interesting that the load method to DfGen1, PBI datasets, and staging for DfGen2, is different to final destination for DfGen2.

[James Fancke @jdfancke]

@GregBaldini @mim_djo @PawarBI I could be misunderstanding, but I'm experiencing the issue as per this article (data loads to Staging but not the final DWH): https://t.co/N2F4qa5gk8

You can recreate the issue by blocking outbound TCP 1433 on the on prem gateway server.

[James Fancke @jdfancke]

@GregBaldini @mim_djo @PawarBI Thanks and that's interesting. My guess of what's happened is the gateway loads to the DataflowsStagingWarehouse over 443, but then when doing the 2nd load to the user made DWH it loads over 1433. The gateway docs don't mention 1433 (I guess cause preview) https://t.co/ocbeOuPpyD

[James Fancke @jdfancke]

@DenSelimovic @kurtbuhler @PawarBI @Kjonge did a great podcast recently with @TedPattison
https://t.co/JYvrw2h6oZ

[James Fancke @jdfancke]

@notaboutthecell @PawarBI @mim_djo @EscobarMiguel90 I'm guessing this is by design and won't change in future (as Dataflows Gen2 is a "Data Factory" artifact instead of a "Power BI" artifact), and ADF flows to Azure SQL/Synapse require outbound 1433. https://t.co/th5YRl4Qid

[James Fancke @jdfancke]

@notaboutthecell @PawarBI @mim_djo @EscobarMiguel90 Turns out I thought 1433 was open, but it wasn't 🤷‍♂️ (probably due to network/proxy firewall rules). Protip for others that ever find themselves in this situation, check https://t.co/MavTSq7Eu8 on the machine with the on prem gateway.

[James Fancke @jdfancke]

@2meterDBA @mim_djo @PawarBI Thanks yes I tried that too. I was hoping turning off staging would mean it loads direct to the destination rather than hopping back to on prem before sending again to the destination DWH (but seems it still does the back and forth).

[James Fancke @jdfancke]

@mim_djo @PawarBI @notaboutthecell @EscobarMiguel90 Jealous! 😲

[James Fancke @jdfancke]

@PawarBI @mim_djo @notaboutthecell @EscobarMiguel90 Ah thank you. I assume you also need inbound ports open on the gateway server as well to read from the DataflowsStagingWarehouse. (I have TCP 1433 outbound open but still getting failures, I'm wondering if it's an issue within an inbound rule)

[James Fancke @jdfancke]

@kurtbuhler I particularly like the "Analysis Services" style (sort of) icon for the dataset