FlexiBERT leverages a novel architecture search technique, BOSHNAS, on an expanded design space of transformer architectures with diverse operations. We propose a suite of models that beat the baseline frameworks.
Are current transformer models too homogeneous and rigid? Read our new preprint to find out -- https://t.co/gXRiLMCAqD! FlexiBERT pushes the performance frontier by expanding the limits of traditional models. @bhish_98@ShreshthTuli