Nils Rethmeier

PhD candidate. continual SSL, low-resource SSL, unsupervised XAI, data-efficient multi-modal learning @Copenhagen University, @DFKI.

[Nils Rethmeier @Nils_Rethmeier]

@micahgoldblum @DamienTeney It makes the review system an adversarial setting -- and larger labs game the system. It's a really bad design/ incentive, and game theory tells us why.
One should not be reviewer in a conference, where one has a paper. esp. not in the same track. That's why arxiv 1st, conf 2nd.

[Nils Rethmeier @Nils_Rethmeier]

When your (model) representations did not turn out quite right. #model_understanding #XAI

[Nils Rethmeier @Nils_Rethmeier]

@micahgoldblum Trusting the mean of 3 random draws from a distribution is statistically ...

[Nils Rethmeier @Nils_Rethmeier]

@rohitrango Hardware weight classes -- boxing analogy. Yes please!!!

[Nils Rethmeier @Nils_Rethmeier]

@tdietterich @Aaroth @rohitrango Is there a good list of 'standard' or wise practices in this regard? Or should one just scan @UAI?

[Nils Rethmeier @Nils_Rethmeier]

@RussellSPierce @wasimlorgat Is that what you meant @wasimlorgat? Because developing in dynamic, debuggable cells and tabnine + interactive autocomplete plus seemlessly going from .ipynb to .py is kinda amazing, but seems to be barely known. Also, works with remote kernels :)

Or did you mean more?

[Nils Rethmeier @Nils_Rethmeier]

@micahgoldblum @sahilsingla47 Yes, there are deals. Depends on where the funding is from. In Germany. Usually the prof is payed and grads are not. If you join an "exzellenzcluster" you may get a program that gives 2 grad students a 3 years funding programs. The program defines the rules, not the Uni afaic.

[Nils Rethmeier @Nils_Rethmeier]

@micahgoldblum @sahilsingla47 But this is rather the exception. Most Profs have to acquire project based funding from the government in competition with other establised institutions, which means you need to work on sth that the gov sees as funding worthy. The "games of names" is key in both paths.

[Nils Rethmeier @Nils_Rethmeier]

@micahgoldblum @sahilsingla47 SOME! Depending on country, there is basic funding, but deciding who gets it is very political. E.g. MPI is german, and if you did not grow into german academia connections you wont be that fundee. Profs try to get into excellency programs, that finance 2 grads.

[Sara Hooker @sarahookr]

A great piece in @VentureBeat by @sharongoldman about what we hope to achieve with @forai_ml.

https://t.co/uMHnHCmT7N

[Nils Rethmeier @Nils_Rethmeier]

@micahgoldblum Or the old. "I'm only convinced if you train a 6B param model during the review phase.". "Oh you did?! Well now train an 6B param Random_NN_architecture_from_2015 for comparison. That will convince me - and I'm not just trying to increase my papers' chances. Promise this time!!!"

[Nils Rethmeier @Nils_Rethmeier]

@ilyasut Perhaps it would help to see how humans get biased, to come up with mechanisms to adjust bias learning in models. The same dual use applies though.

[Nils Rethmeier @Nils_Rethmeier]

@BlackHC @roydanroy Vaguely recall Shannon saying that he did not mean IT to become a final theory, but a first bet into a direction, but that people overfit to the first version. Can't find the ref. tho. Are you aware of any such improvement theories over IT?

[Nils Rethmeier @Nils_Rethmeier]

@rasbt @kskrygan @EmreSevinc VS code quick list
+ remote: https://t.co/n5riISEkVw, there is a containers version too
+ interactive .py, aka dynamic notebook mode: https://t.co/M8Swva9ecE
+ VS code on Colab: https://t.co/44jlZagvFL e.g. by @abhi1thakur
+ on iPad (early-ish): https://t.co/h3Psnps9Kq

[Nils Rethmeier @Nils_Rethmeier]

@rasbt @kskrygan @EmreSevinc VS code has remote dev since long now (maturity) + works on G-Colab + iPad. VS also has mixed .py interactive (aka notebook) mode (works remotely) -- for free. In PyCharm it's/ was a payed feature. Also VS code is lighter, and due to less clutter easier to learn for trainees.

[Nils Rethmeier @Nils_Rethmeier]

@ilyasut According to an older (2004) paper the same brain structures that process emotions, when damaged, largely inhibit decision making. So in a ways yes: rational = emotional wrt. human brain architecture. https://t.co/MofMLw4MJP

[Nils Rethmeier @Nils_Rethmeier]

@ani_nenkova IMHO a 4 pager takes half the time and depth an 8 pager takes. So 4 pagers don't necessarily stack well against 8 pagers and often only present comparably superficial studies. Often 8 pages already limit a paper to 1 or 2/5 necessary aspects, and the *ACL.sty shortens further.

[Nils Rethmeier @Nils_Rethmeier]

@BlackHC Some recent methods go away from probabilistic, which is arguable less restrictive. Let's see how that will be bayesplained post-hoc :)

[Nils Rethmeier @Nils_Rethmeier]

@philipvollet Would be curious to know how this compares against other neural topic models (NMTs), i.e. models in https://t.co/AqwJ22o4R4, which have similar advantages.

[Nils Rethmeier @Nils_Rethmeier]

@thoward37 @soumithchintala Similarity (STS/ search/ recommendation) oriented Transformers somewhat, but BERT no. It's actually hard to beat BM25 (TF-IDF) consistently and TF-IDF somewhat allows the user to influence the results by re-keywording a query.