Transformer-Based Biomedical Text Extraction

Unlike formal languages, natural languages are unstructured and more complex. Understanding and generating meaningful text are the goals of natural language processing (NLP). Deep learning recently had a significant impact on this field. Innovative and effective transformer-based models have achieved state-of-the-art results on a wide range of NLP tasks, including those working on specialized clinical and biomedical text. The most widely-adopted models (BERT and GPT) are here described along with their domain-specific versions and applications in the biomedical domain.