Master thesis: Evaluation of multi-tool and multi-agent systems

ericsson | Telecommunications

2h 17m ago

On-site

Full Time

Sweden

Join our Team

About this opportunity

Enterprises are now having many agentic systems with multiple agents interacting with each other and multiple tools being available to the agent for use. These tools are a mix of deterministic systems, vector embedding search, as well as generative models. They may be multimodal covering text, image, graph, code, etc. in both input and output.

The main challenge is evaluation of such systems esp. given the non-deterministic nature and often the large number of tools available. Additionally, for tools which are probabilistic / vector search multiple results are returned some of which are wrong. These problems become pronounced for specific tasks, like in problems in telecommunication systems where preciseness of answer is very important.

What you will do

Generate a master set of questions for agentic evaluation covering a wide range of topics, types and complexity
Create an automated logging framework so that every tool call’s input and output is logged
Evaluate the system input/output with multiple runs of master set of questions and observe patterns between runs and across questions
Check on LLM-as-a-judge evaluation and see possibilities and how to calibrate scores

The skills you bring

Background in EE/Computer Science/Mathematics or related
Familiarity with wireless communications / signal processing
Proficiency in Machine Learning and deep learning 
Good programing skills - C++, Python 
Familiarity with programming environment in Linux

Why join Ericsson?At Ericsson, you´ll have an outstanding opportunity. The chance to use your skills and imagination to push the boundaries of what´s possible. To build solutions never seen before to some of the world’s toughest problems. You´ll be challenged, but you won’t be alone. You´ll be joining a team of diverse innovators, all driven to go beyond the status quo to craft what comes next.

What happens once you apply?Click Here to find all you need to know about what our typical hiring process looks like.Encouraging a diverse and inclusive organization is core to our values at Ericsson, that's why we champion it in everything we do. We truly believe that by collaborating with people with different experiences we drive innovation, which is essential for our future growth. We encourage people from all backgrounds to apply and realize their full potential as part of our Ericsson team. Ericsson is proud to be an Equal Opportunity Employer. learn more.

Primary country and city: Sweden (SE) || Stockholm

Req ID: 790724

Required Skills

mldeep_learningc++pythonlinux

Required Languages

🇬🇧 English