For speech, it was superior to Meta’s own wav2vec 2.0 self-supervised speech algorithm, while for text it was tested on the GLUE benchmark suite and performed as well as BERT, another commonly ...