awkhub

@awkhubio

usage: awk [-F fs] [-v var=value] [-f progfile | 'prog'] [file ...]

[アカギ｜AI×プロダクトの最前線を毎朝発信 @kenta_akagi]

従来のRAGは埋め込みモデルで近似検索するから、完全一致・スパースな手がかりの組み合わせ・多段階の仮説検証が構造的に苦手だった。DCIはgrepやawk、Pythonスクリプトで生テキストを直接操作。ChromaやPineconeを挟まず、エージェントがシェルレベルで検索ロジックを自分で組み立てる設計。ベクトルDBが丸ごといらなくなる可能性がある。

[Ming "Tommy" Tang @tangming2005]

9/
This tells awk:
Every time chromosome changes, close the previous file.
Then write the new line to a file named like "1.bed", "2.bed", etc.

[FreeBSD Frau @freebsdfrau]

@gumnos @lordbaco @awkhub Just as a semi-related aside: did you know you can have as many BEGIN/END blocks as you like? They will be executed sequentially

[Ming "Tommy" Tang @tangming2005]

5/ Example: AI writes you an awk command to parse FASTQ.
It runs fast. But it trims the wrong bases.
Without knowing awk, you trust junk data.

[Goksal @goksaladiguzel]

@tangming2005 @awkhub in the past, since we had limited resources, we tried to use as few commands as possible; the line below will do the same job, and you can measure the elapsed time using the time or timex command.

awk '!/^#/ {print $1 "\t" $2 "\t" $5}' *.vcf | sort | uniq -d | wc -l

[Erkki Ruohtula @erkkiruohtula]

@igor_os777 Awk is actually pretty straight-forward compared to most newer languages (Python included). Easy to learn.

[ポエムコード @poem_code_jp]

「AIがあればAWKはオワコン」
そう思ってた。

10億行ログの集計をAIに頼んだら
トークン上限で死んだ。

awk '{sum+=$3} END{print sum}'
3秒で終わった。

#AWK #AI #Shell #PoemCode

[tac0ghost 👻🇺🇲👻 @tacooghost]

@igor_os777 awk - the tool I've been using for 30 years but still don't completely understand

[Igor Os @igor_os777]

awk sticks around for a few very practical reasons, even if it looks like someone compressed a programming language into a crossword puzzle.

First, it fills a gap nothing else quite hits. It lives right between quick-and-dirty shell tricks and dragging in something heavier like Python. If you just need to grab a few columns, total some numbers, or reshape a text file, awk does it instantly. No imports, no setup, no extra files sitting around afterward. One command, done, move on with your life.

Second, once it clicks, it actually makes a lot of sense. The whole idea is simple: “when you see this, do that.” That’s not some exotic paradigm, that’s just how people think. The trouble is the syntax looks like it lost a fight with punctuation. Push through that phase and it starts to feel pretty clean.

Third, it’s everywhere. Any halfway normal Unix-like system already has it. Locked-down servers, stripped-down containers, random ancient boxes in a corner rack, awk is just sitting there like an old wrench that never got thrown out. You don’t have to ask permission or install anything, which is half the battle in real environments.

And then there’s the accumulation effect. awk has been around forever, so it’s baked into scripts, Makefiles, and decades of forum answers. You run into some cryptic one-liner, stare at it for a minute, figure it out, and now that trick is yours. Repeat that enough times and suddenly you know awk whether you meant to or not.

It’s not pretty. It’s not friendly. But it keeps getting the job done, which is more than you can say for most “modern” replacements that show up, trend for a year, and quietly disappear.

[Liz Nonyabiz @nonyabiz_liz]

@igor_os777 I like #AWK or #GAWK

you can do scripts with the shebang
#!/usr/bin/awk -f
your script here...

very 'C' like but looser & scripted

handles floating point calculations & regex though only has rudimentary higher math

😼

[Jordi Portell @yshalf]

@igor_os777 Awk is amazing. I implemented some tools to swiftly handle huge datasets from the Gaia mission (billions of records in CSV files). I even managed to run a rudimentary clustering algorithm in a simple PC on that huge set. Even a SQL-like tool for CSVs:
https://t.co/bHVzWseIQ5

[Ming "Tommy" Tang @tangming2005]

1/ Just hit 60K followers on Linkedin. Was Googling basic awk syntax, now ask AI to do it for me.

Was reading other people's code to understand, now use AI to teach me.

The only difference is now 60,000 people watch me do it.

[S0AndS0.eth @S0_And_S0]

@Jason_Ewton @DThompsonDev Recently reviewed AI _audit_ I had 100% hallucinated bugs plus zero QA by human :-|

For replacing strings, complex or simple, I've found `awk`, `sed`, or `vim` are trust-able and testable.

Willing, ecstatic, I'd be to see evidence of AI productivity enhancements being real

[nel @McNEL608]

@jaredlander i would have probably just piped the csv into an awk script

[Ming "Tommy" Tang @tangming2005]

3/
Here it is:
cat *.vcf | grep -v '^#' | awk '{print $1 "\t" $2 "\t" $5}' | sort | uniq -d | wc -l

Let’s break it down.
Line by line. Word by word.

[Ming "Tommy" Tang @tangming2005]

🧵 Mastering AWK for Bioinformatics: Data Wrangling Made Simple
AWK is a powerhouse for genomics data manipulation. Here’s a quick guide with practical examples 👇

[Psychofren @psycho_fren]

@cemaxecuter @thebaldgeek a bit ghetto but this one liner works to dump unix timestamp and temperatures for processor cores:

echo "$(date +%s) $(sensors -u | grep -A4 'Core' | grep '_input' | awk '{ print $2 }' | tr '\n' ' ')"

1753931199 32.000 32.000 33.000 32.000

[CraftedStack @Crafted_Stack]

Have to do some analysis on files but only have terminal access?
awk is your best friend!
Here is a one liner which counts the frequency of each unique line

awk '{count[$0]++} END {for (item in count) print item, count[item]}' file.txt

[awkhub @awkhub]

https://t.co/h7qaDPM4nP

[Dmitry Matveev @DmitryMatveev]

Just wrote a simple bash loop with curl streamed right into a simple awk script's stdin to parse and format a summary of ~15 Jenkins jobs (250 MB logs total) into a neat nice little table. What a joy, this is what vibe coding takes away from you