The predecessor of the Language Technology Research Group, the Corpus Linguistics Department, was founded in 1997, as a formal recognition of the language technology research and development work that had been going on for several years at that time. Since then, the research group has gained significant experience in research and development in many areas of language technology: it has achieved outstanding results in building language resources, developing language technology tools and, more recently, in teaching language models.

In terms of linguistic resources, the first version of the Hungarian National Corpus (HNC or MNSZ)  must be highlighted: the 187.6 million-word annotated, representative text corpus, completed in 2005, which also includes language variants from territories outside the present borders of Hungary. It was the first major database of its kind for the Hungarian language. MNSZ2, the improved version, published in 2014, not only contains almost ten times more words (1.5 billion), but also covers new, important text types, such as social media. Furthermore, the quality of the linguistic analysis also improved a lot compared to the previous version.

Over the years, the members of the research group have developed a number of tools. One of the most significant of these is the helyesiras.mta.hu – the spelling service portal, which was created to help language users in improving spelling, following the complex rules of the Hungarian spelling system, using automatic tools. e-magyar, the digital  language processing toolchain and its improved, modularized version, emtsv, which enable extensive analysis of texts should also be emphasised.

The Hungarian version of the WordNet lexical database is another important resource, which was created with the cooperation of the research group. HuWordNet was created as a result of a three-year- process and depicts the Hungarian vocabulary from a semantic point of view: it includes both synonymous words and the relations between them.

The scientific paradigm shifts that took place in 2013 and then in 2018 had a serious impact on the work carried out in the research group. Following the leading international research, we created the Hungarian versions of the neural language models developed primarily for English. Initially, this meant static word embeddings, but now we also have several transformer-based contextual language models. Examples include HILBERT, which is a BERT-Large language model, and PULI-GPT-3SX (7 billion parameters), the Hungarian version of GPT-3.

The central focus of the research group’s current activities is the development of instruction-following models: within this framework, we have created the ParancsPULI and  PULI LlumiX 32K Instruct models. Other specific applications related to the language models can be tested on this demo page.

Another priority of the research group is to create test databases for Hungarian, so-called create benchmark corpora that, when embedded in a web service, enable a simple, yet multi-criteria evaluation of neural network-based technologies, as well as the comparison and publication of these results. For this purpose, the Hungarian Language Understanding Evaluation Benchmark Kit (HuLU) was created, based on the GLUE and SuperGLUE test database infrastructure developed for the English language. Additionally, procedures for evaluating generative language models are also being developed.

Research Group Leader: Enikő Héja, PhD Email: urwn.ravxb@alghq.uha-era.uh Phone: +36 (1) 3429372 / 6043





*A detailed list of the closed tenders can be found here.