Top Tweets for #GridEngine
@freebsdfrau You seem to have solid knowledge of #GridEngine. Are you interested in joining the team working on its successor, #OpenClusterScheduler? We just released our first version...
I'm delighted to announce that my journey with #GridEngine continues at @HPC_Gridware. The product has a new name: #OpenClusterScheduler. Find out more here: One product, three people and a handful of companies https://t.co/kiW6CURs7I
TIL Sun #GridEngine (SGE) supports something called a “jobnet” wherein you qsub a job and get the job ID and then submit another job with “qsub -hold_jid LIST” to make the second job wait before LIST of jobs are complete before starting, and this has me thinking… 🧐
Hidden in the Sun #GridEngine test-suite is a utility NOT installed by default named qevent which allows you to tap into cluster job events. I used this to create a new cluster utility that can wait for a job on the cluster to finish before executing a command
12 months after that, on January 18, 2011, Univa announced it had acquired from Oracle the core Sun #GridEngine developers including the CTO https://t.co/WUdxnYNdhE
2 months after Roland decided that this experimental feature (now ~4yrs old) needed to go, he sent out one single e-mail to the #GridEngine mailing lists, asking if anyone uses it before he wields his axe https://t.co/wLSFHYfwTE
Hop in the time machine with me, and let's travel to the year 2004, a hair over 16 years ago. A Sun Microsystems employee by the name of Stephan Grell had been working on an experimental feature to improve the performance of #GridEngine dramatically and on July 30, he commits it!
Some schedulers thought they would out-perform decades-old Sun #GridEngine by using an optimization, but little did they know that SGE contained the same exact optimizations (off by default and unstable) more than 10 years earlier https://t.co/PyBuKr4rNH
Show you the code? OK ... https://t.co/jJceRtzjaB
Still interested in how a decades-old job scheduler might have contained the necessary bits to dethrone even future contenders?
Stick around and we'll see how Sun #GridEngine may just surprise you
I am releasing details today. All information has been gathered, timelines analyzed, old code vetted, major flaw fixed, and results scrutinized. Strap-in boys and girls, this rollercoaster goes to 11! Sun Microsystems #GridEngine has been hiding an ace 🂡 up its sleeve for 16 yrs
Details still pending, but what if I told you I discovered a way to improve cluster scheduling performance under load by up to 60000% or more? Sun Microsystems hid a really amazing optional feature in #GrodEngine and it has largely been unknown/unused/unfinished for 19 years!
Just a quick reminder, last week I got diverted from #GridEngine optimizations to work on real-time I/O metadata monitoring. I’ll be finishing that up soon and returning to SGE improvements
It turns out that the LIGO-Virgo sattelite data is processed using the same software that I maintain at work. Like me, they discovered that #GridEngine has limitations. They made enhancements to support millions of jobs per month on an HPC cluster of 35,000 cores
@gerbrandvd Yes. Here is a copy of the original Sun #GridEngine source, specifically the code comprising the hidden feature. Hidden in plain sight with only a couple terse references in two places that would be easily missed https://t.co/lczHVJSsAc
And so, I will be cleaning up the patch and releasing it publicly. This excites me because SGE is also used for things like processing CT scans. Imagine if your hospital is running an unpatched #GridEngine in default configuration. As more CT scans need to be processed, ...
But I save the really really good stuff for myself. That's right, I make dashboards for myself too. This is what it's like to watch a MM$ cluster scheduler in realtime. I mean, Airflow has DAGs, but Sun #GridEngine with my enhancements blows static images away
Pretty sure that peak of ~11k jobs was me trying to crash the cluster scheduler because we are in the midst of optimizing Sun #GridEngine for handling >5M jobs per-day and in-doing-so we had to customize the C code to eliminate inefficiencies in job scheduling
Let's have a look at some of the dashboards that I am most proud of. Here we see the dashboard for monitoring Sun #GridEngine queue states. Each node in the cluster can be in 5 abnormal states. Error, Unreachable, Alarm, Disabled, and Suspended. This dashboard sends alerts.
With 2 hours shifts I need to set up Grid engine for myself to be able to parallelize all the work that needs to be done at the bench!
#bioinformatics #gridengine #stanford #benchwork #postdocduringlockdown
If you are running an #HPC cluster and use #GridEngine software (Sun, Open, Oracle, or
@Univa_Corp) you will probably want to check out my new *GE enabled Linux kernel tracing scripts that can monitor VFS I/O on a per-job basis
https://t.co/cczPBkam0H
https://t.co/gQGn8qhjV2
Last Seen Hashtags on Sotwe
AuthorLife
Seen from United States
女爹
Seen from United States
รับงานนครนายก
Seen from Thailand
beastars
Seen from United States
sonandmom
Seen from United States
NOlimit filter:videos
Seen from Turkey
オナ
Seen from Japan
nolimit() #momson() +filter:native_video
Seen from Brazil
niğdeCd
Seen from Turkey
Teenager
Seen from France
Most Popular Users
Elon Musk 
@elonmusk
240.7M followers
Barack Obama
@barackobama
119.2M followers
Donald J. Trump
@realdonaldtrump
111.7M followers
Cristiano Ronaldo
@cristiano
110.9M followers
Narendra Modi
@narendramodi
107M followers
Rihanna
@rihanna
97.7M followers
NASA
@nasa
92.2M followers
Justin Bieber
@justinbieber
90.9M followers
KATY PERRY
@katyperry
87.8M followers
Taylor Swift
@taylorswift13
81.7M followers
Lady Gaga
@ladygaga
73.2M followers
Virat Kohli
@imvkohli
70.1M followers
Kim Kardashian
@kimkardashian
69.9M followers
YouTube
@youtube
68.7M followers
Bill Gates
@billgates
64M followers
Neymar Jr
@neymarjr
62.9M followers
The Ellen Show
@theellenshow
62.4M followers
CNN
@cnn
61.9M followers
Selena Gomez
@selenagomez
60.9M followers
X
@x
60.8M followers