context4ai

Context4ai is a thinktank, applying AI to create social value & contribute to collective joy. | Founders: @olasofia, @evangineer. What a time to be a polymath!

[context4ai @context4ai]

@simonw For me, key features are autonomy and having a mission / purpose as an aim.

[context4ai @context4ai]

@Teknium1 @abacaj

[Hao Zhang @haozhangml]

Our latest work on long sequence training (almost) reinvented sequence parallelism with many new optimizations very specific to today's decoder LLMs and memory-efficient attention.

Training 2x faster on 8x longer sequences!

There is a secret trick that can readily accelerate the training of many LLMs by 10% (regardless of the seq length) if you are already using memory-efficient attention, go find it out in the paper 🧐!

Kudos to our amazing students: @DachengLi177 @RulinShao and Anze!

[context4ai @context4ai]

@erhartford @Teknium1 So with you on this!

[context4ai @context4ai]

@Teknium1 I am waiting for @the_bloke's Discord server to get referenced in a paper. It's just a matter of time!

[context4ai @context4ai]

@bbbozzz @ClementDelangue Yes, it's like saying there is no point to having the US Constitution or the UN Declaration of Human Rights.

It's an argument made in bad faith.

[context4ai @context4ai]

@tophinity

[context4ai @context4ai]

@mustafasuleyman We know that you are smart enough to know that the problem is one of governance.

So it feels disingenuous of you to point at open source, naive or otherwise while ignoring perverse incentives current political economy provides to corporate actors with their governance failures.

[context4ai @context4ai]

@abacaj The $1.3 bn dollar question.

Language feels like Bill Joy's The Future Doesn't Need Us. Calling out Open Source seems to be from BigTech regulatory capture playbook. Not acknowledging the default outcome of monopolised markets signals intent to become one of the monopolists.

[context4ai @context4ai]

@icreatelife It's a classic. Read the original Isaac Asimov novella in my teens.

[context4ai @context4ai]

@HamelHusain I imagine the surge in demand is being driven primarily by Llama 2 finetunes, given what I've been seeing.

[context4ai @context4ai]

@abacaj @ledger_luminary Exactly. Finetunes of the smaller Llama 2 models is within scope of the license.

Rewoo is a good example of what can be done.

[context4ai @context4ai]

@bio_bootloader @leotarla Personally, I am interested in Replit, Starcoder and Llama 2. I will take a look at your code and see what I can do.

[context4ai @context4ai]

@MikeYearworth @evgenymorozov Feels very much in line with the Douglas Engelbart vision of Intelligence Augmentation.

A sociotechnical approach.

[context4ai @context4ai]

@s3nhs3nh @abacaj GPTQ is a different quantization format from GGML.

[context4ai @context4ai]

@abacaj Yes Starcoderbase looks ideal for finetuning.

[context4ai @context4ai]

@tomaskafka @random_walker @goodside I was thinking something very similar.

[context4ai @context4ai]

@SashaMTL You're not wrong. We need Citizen Assemblies to frame the issues and formulate the solutions and the political will to act accordingly.

[context4ai @context4ai]

@JagersbergKnut @labloke11 @SashaMTL I have observed that even updating the readme triggers a refresh.

[context4ai @context4ai]

@icreatelife Y of course!

[context4ai @context4ai]

@CopySutra @JeremyNguyenPhD Having a decentralised AI model is going to be so important.