Improving BERT Pretraining with Syntactic Supervision
A proof-of-concept work with promising results on embedding syntactic biases onto bidirectional transformers with no compute overhead.
A proof-of-concept work with promising results on embedding syntactic biases onto bidirectional transformers with no compute overhead.