Parth

Programmer | Distributed Systems | Applied AI

[Parth @parthlogs]

@ChShersh where exactly? that first push

[Parth @parthlogs]

@AI_Edge_Studio @UiSavior tell me more

[Parth @parthlogs]

@AI_Edge_Studio @UiSavior hmm it's truly taxing, the inheritance of frontend assumptions in backend tooling.

[Parth @parthlogs]

@Ishansharma7390 It's promising at best

[Parth @parthlogs]

@AI_Edge_Studio @UiSavior exactly, so preposterous of them lol

[Parth @parthlogs]

@amitwt @Hi_Mrinal Ofc 😅

[Parth @parthlogs]

@amitwt @Hi_Mrinal All genuine questions 😐

[Parth @parthlogs]

Ah finally! @nikitabier I take it as my moral responsibility to do my bit towards keeping this platform free of junk. Thank me no thankings.

[Parth @parthlogs]

@Hi_Mrinal Hehe 😅 the getting old part is a hard pill to swallow. You can definitely accelerate your growth but there's a cap to that speed, you cannot outrun time

[Parth @parthlogs]

It is not *funny* anymore. It's exhausting.

[Parth @parthlogs]

It's funny how everyone is continuously oscillating between the Apocalyptic AI and the unreasonably hyped AI

[Parth @parthlogs]

For anyone who has coded even a little, it is obvious which parts of code are tedious and boilerplate and which ones are interesting/custom to the project.

With that understanding, it should be clear when to use AI and when not to. Why is there so much amnesia around this topic? Why do we have polar opposites?

AI adoption isn't even a thing worth contemplating. Just swallow it like any other developer tool or abstraction you have swallowed (and normalised) already!

[Parth @parthlogs]

Absolutely! Reports/papers on past failures and incidents are the best learning resources for any systems engineer. It is also very assuring since we acknowledge the fact that even the big players make both - trivial and catastrophic mistakes.

Twitter's 'Fan Out' architectural problem or the infamous Netflix ELB cascading failure, etc teach far more than any book or lecture would.

[Mrinal @Hi_Mrinal]

TBH there is soo much untapped learning hidden in technical reports by tech companies which they release after a major downtime .....

these three were my recent reads

[Parth @parthlogs]

@yacineMTB Who said you can't autodidact through depth or cutting edge

[Parth @parthlogs]

@Hi_Mrinal U 2

[Parth @parthlogs]

@sh_reya How do you arrive at the dimensions themselves? Domain expert work?

[Parth @parthlogs]

I am particularly curious about evals for startups at stages where they don't have traces at all yet - unlike examples where you can evaluate conversations already held by AI.

This could mean pre-production products, or apps where the nature of the response is completely different from chat.

What would an evals solution look like for a startup that is still deciding the model, params, prompt, and context?

Surely, different decisions here can yield radically different outputs.

The obv solution that comes to my mind is to generate synthetic/manual representative cases and run a configuration tournament across model + params + context combinations, while accounting for things like position bias and other similar mathematical aspects.

Is there a better way to think about evals before real traces exist? Curious about how @HamelHusain and @sh_reya would think about this

[Parth @parthlogs]

@HamelHusain hmm. I'm going to test this properly: seed evals with synthetic/manual cases, then run a tournament across model + prompt + context + cost configs, with judge-bias checks and confidence intervals. Will share concrete numbers once I have them.