langroid

An open-source Python framework that simplifies LLM application development using a multi-agent programming paradigm.

Langroid

Langroid is a multi-agent framework and seems to be a fresh take on all of these large language models and agente software. Langroid is an intuitive lightweight extensible and principled python framework to easily build llm powered applications from X C CMU and UW medicine researchers. So there is a very thorough educational and academic background behind this project. 

In this video we are going to install this Langroid locally and then we will see how can we directly chat with llms. Plus we will also be looking at building AI agents with Langroid. Before I show you the installation and we go deeper into this Langroid, let me give a huge thanks to Mast compute who are sponsoring the VM and the GPU which I'm going to use for this video. 

If you are looking for a affordable GPU option then I will drop the link to their website plus a coupon code which will give you 50% discount on an already affordable pricing on range of gpus. Now coming back to this Langroid, so as I said that you can set up agents equipped with um optional components like llm vector store tools functions assign them tasks and have them collaboratively solve a problem by.

Exchanging messages this multi-agent paradigm is inspired by the actor framework. But you don't have to really worry about this actor framework. It is a very interesting research by the way, and I have done um I think one video on it last month. 

So all in all, actor model is primarily a mathematical model of concurrent computation that treats an actor as the basic building block of concurrent computation. In response to a message it receives, an actor can make local decisions, create more actors, send more messages, and determine how to respond to the next message received. 

Don't worry, I'm not going to go into more detail of this actor framework, but very interesting if you want to read through it. I will drop the link uh to the repo of Langroid in video description. Plus, you can also get the actor framework link from there. 

Anyway, so um let's try to get it installed first and then we will see how the cookie crumbles. Let me take you to my VM here. I'm running 12 22.4 and the GPU card from M computers, this Nvidia RTX A6000. For this video, you don't need that much VRAM because I will be using it with OpenAI just to show you the workings of.

Langroid but in the future videos I will be using Langroid with AMA and local model so stay tuned. And for that one you would need of course a larger GPU with VRM. Let me clear the screen and this this is a K environment which I will be creating with K 24.4.2. That is done. Let's clear the screen and let's look at it. It's huge. So but don't worry about it. It uses a lot of stuff which seems quite familiar anyway. 

Now let's use pip install Langroid. I always go with all because it installs Langroid plus its integration. Okay, so when you receive this PG config one, don't really worry about it. All you need to do is to simply run it like this CU. It depends on the operating system and I have seen it happening a lot of times. So instead of using this, I'm just going to go with Langroid and I will install the PG vector or Postgres vector separately. 

So let's wait for this one to finish and then we will install it. We don't really need that one for this video but I will show you how you can get around that one. Let's wait for it to finish. The Langroid got installed. Let me clear the screen and for that error all you need to do is to install this P install s copy G2.

Binary and then it is going to install it locally here and that is done and you should be all good to go. 

Okay next up let's create the environment file by copying the template if it is not there that is fine maybe I will just go. 

Okay I didn't go into it that is the problem. 

Okay so let's do it now now this is good and now let's open this enem file. 

So here all you need to do is to just replace this open a key with your own key and you can get that key from platform.

Open.com and that is a paid option. So once you set it, I will save it and then clear the screen. So let me quickly do it. So I have set my key, all is done. 

Let's launch our python interpreter and then play around with this Langroid. Let's first import Langroid. I'm just calling it alas as LR and then language model from Langroid as LM. 

And now let's set up our model for the model configuration. I'm just using lm.

Openi with openi chat model and then we can also create the model directly something like this. And you can talk with model very very easily. I'm just using the chat one. 

Okay, so it says said it couldn't find it error in open your GPT Cash Store. Okay, okay, I think I didn't put in the codes there with my API key in the environment file. Let me do it, but I think I can ignore it because it really gave me the answer. I'm just going to ignore this then for now. You see I just asked it what is the capital of New South Wales. It has given me the answer here. 

So, and I'm I think this is to the I didn't put the codes in. Maybe let me put in the codes and rerun it just to see if my theory is right. And there you go. So I have put in the codes around my API key plus what I did I just commented the reddish part in the EnV file. 

And you see it is giving me now the warning about redus, but we are not using that um memory store redus. So I'm just ignoring this warning. So you see we got the answer at least. 

Okay, so now we know how to chat with our llms using this Langroid. And then let's use this llm in an agent. Let's define the agent configuration first and I'm just.

Calling it agent _ CFG passing this llm. Let's also grab the agent on the basis of this agent CFG. And then let's ask the agent a question. I'm asking it what is the capital of Australia. These are some of the token stats, and then the agent has come back with the response here. 

And then we can talk further with it. You can just say agent llm, okay? And Fiji, it has given you the capital of Fiji here. And then, of course, you can programmatically use this as easily as you like. 

Now we can also wrap the agent in a task to run an interactive loop with the user or even other agents. So let me press enter a few times. Let me create the task with it because if you look at it, what exactly these agents are, they perform some tasks. 

And tasks in return are called the tools, which are the functions. So I am defining the task that agent name is bought, and my system message for llm is that your helpful assistant. And then you can just give it any task. I'm just giving it hello, and then it is starting the agent bot. 

It is giving with the answer, how can I assist you today? Similarly, let's define a teacher agent, and let me clear the screen first.

And in order to clear the screen before we need to exit it because you see we in a chat loop with the agent. You can even talk with it. Maybe let me show you quickly. You see I am just talking with that agent. Um can you you create a file uh test.txt in present working directory? Okay okay so it is not telling me it is not creating because it doesn't have that tool. Look at the response. Okay that is good and now you can just simply type Q here and you can exit. 

Okay now in order to clear the screen I normally use this Lambda function in Python. So and the screen is cleared now. Now let's create a teacher agent Loop similar on the similar ground. So I'm just defining this agent configuration. Let's define the task for this agent and the agent ask it teach is a teacher. No system messages ask your student concise number question and give feedback. Start with a question and then let's also Define a student. Student is defined and then you can also Define the task for the student. 

And let's make teacher to ask a question from the student and then we will run the teacher's task and then it will go into the flow. There you go teacher ask this and then you.

Can even talk more what is the sum of this. You can um just say I don't know. Okay, so now teacher is saying let's work through it together. It says what you like to try another uh and can we try playing some game instead. Uh can we play um soccer. 

Okay, you you see you can talk with this teacher and this has just become a loop. But if you look at the big picture what is what is happening is that we have just created a multi-agent application whereby using these LLMs without providing much data they're able to talk with each other and then go from there. 

So really like this um project Langroid and as I said earlier we are just using it with OpenAI at the moment. But in the future video I'm going to show you how you can use it with AMA. So stay tuned and share AI Agent Store with colleagues. I hope that you enjoyed it. Thank you.

Note, that there may be slight differences in text and video. Watch video for the original content.

Langroid Video