Large Language Model AIs like Google’s Gemini and Open-AI’s GPT can be interesting to play around with even in a simple chatbot like Chat-GPT. But those chatbots largely waste their potential. Their understanding of natural language is impressive, but their “knowledge” is limited to what they were trained on. But that’s not the limit of their abilities. By incorporating other data such as external APIs, they can be made much more reliable and effective. To demonstrate this, I used Google’s Gemini LLM to build a bot designed to use the US government’s API to get information about Congress available here.
Accessing the Congress API
The first step was to write a few functions to access the government’s API. I added entry points to list bills, get a bill, and list members of Congress. There are a lot more endpoints in the API, but the good news is that we don’t need to create functions for them explicitly. Responses from the API contain links to related resources, utilizing a common practice in REST APIs. Using these, the agent can navigate the API without being explicitly told about them.
There is one exception, getting the bill text. While most URLs point to JSON objects under the API, these point to external resources in either PDF, XML, or HTML formats. So I made a separate method to get the text.
Then to expose those functions to the agent, I just had to write schemas which I could pass to it detailing how to call the functions. This is a case where you want to be very explicit in how to use the function so that the LLM can understand the situations each function can help with and how to call them.
The Agent
The next step is to write the agent. There exist plenty of tools to build up agents, but with one this simple I just stuck with a single class. It sets up a connection to the Google Gemini model I’m using, give the LLM a prompt on how to operate, tells it what functions it has available, and then loops over the user’s input. Here is the run method
def run(self, commands=[]):
contents = []
while True:
if len(commands) > 0:
s = commands[0]
commands = commands[1:]
else:
s = input("-->")
if s == 'q':
return
contents.append(types.Content(role="user", parts=[types.Part(text=s)]))
resp = self.gen_content(contents)
while resp.text is None:
tool_call = resp.function_call
contents.append(types.Content(role="model", parts=[types.Part(function_call=tool_call)]))
function_name = tool_call.name
function = self.functions[function_name]
args = tool_call.args
print(f"-- calling {function_name}(**{args})")
result = function(**args)
result_part = types.Part.from_function_response(name=function_name,
response={"result": result})
contents.append(types.Content(role="user", parts=[result_part]))
resp = self.gen_content(contents)
print(resp.text)As mentioned before, it loops while getting input from the user until it gets the ‘q’ command to quit. It keeps track of the conversation in the contents array, to which it appends the query from the user. It then sends that conversation to the LLM to get a response. If that response is just a text response, it prints it out and continues the loop. But since we’ve given it some local functions to utilize as tools, we also have to handle it requesting to use one of them. In those cases it will return the function name as identified in the schema along with the arguments it wants to call. We call the function for it and append the results to the conversation, and call the model again. It might want to use multiple tools so we repeat until we just have a text response.
Putting this all together we can use the bot to explore the API and learn about what is going on in Washington.
--> Who represents North Carolina's second district?
-- calling get_members(**{'state': 'NC', 'district': 2, 'current': True})
The member for North Carolina's second district is Deborah K. Ross.
--> What bills has she sponsored recently?
-- calling call_endpoint(**{'endpoint': 'https://api.congress.gov/v3/member/R000305?format=json'})
-- calling call_endpoint(**{'endpoint': 'https://api.congress.gov/v3/member/R000305/sponsored-legislation'})
Here are some of the bills that Deborah Ross has sponsored:
* H.Con.Res.28 Expressing the sense of Congress that the votes of overseas servicemembers must be counted and honored as required under the Uniformed and Overseas Citizens Absentee Voting Act.
* H.R.2886 To amend the Outer Continental Shelf Lands Act to withdraw the outer Continental Shelf in the Mid-Atlantic Planning Area from disposition, and for other purposes.
* H.R.2079 Insurance Fraud Accountability Act
* H.R.10326 Safer Sports for Athletes Act of 2024
* H.Res.1575 Expressing support for designating the week of November 4 through November 10, 2024, as "Forensic Nursing Week".
* H.R.10095 Insurance Fraud Accountability Act
* H.Res.1529 Expressing support for the designation of October 1 as "National Science Day".
* H.R.9849 Medical Laboratory Personnel Shortage Relief Act of 2024
* H.R.9720 AI Incident Reporting and Security Enhancement Act
* H.R.9737 Secure A.I. Act of 2024
* H.R.9580 To designate the facility of the United States Postal Service located at 2777 Brentwood Road in Raleigh, North Carolina, as the "Millie Dunn Veasey Post Office".
* H.R.9443 Federal and State Food Safety Information Sharing Act of 2024
* H.R.9170 Advancing International and Foreign Language Education Act
* H.R.8077 Closing Bankruptcy Loopholes for Child Predators Act of 2024
* H.R.7339 To make revisions in title 51, United States Code, as necessary to keep the title current, and to make technical amendments to improve the United States Code.
* H.R.7211 Petfax Act of 2024
* H.R.7095 Redistricting Transparency and Accountability Act of 2024
* H.R.6309 Understanding Student Parent Outcomes Act of 2023
* H.R.5576 Protect Working Musicians Act of 2023
* H.R.5258 Protection and Advocacy for Criminal Legal Services Act
--> Tell me about H.R.9737
-- calling call_endpoint(**{'endpoint': 'https://api.congress.gov/v3/bill/118/hr/9737?format=json'})
The Secure A.I. Act of 2024 was introduced on 2024-09-20. It was referred to the Subcommittee on Cybersecurity and Infrastructure Protection. The bill's primary policy area is Science, Technology, Communications.
--> What does the bill's text say about why it is important?
-- calling get_bill_text(**{'congress': 118, 'billType': 'hr', 'billNumber': 9737})
The bill says it is "To improve the tracking and processing of security and safety incidents and risks associated with artificial intelligence, and for other purposes." It aims to address risks of physical or psychological harm, threats to human life, health, or property, and the confidentiality, integrity, or availability of an artificial intelligence system.This works fine if you are asking about a specific Congressperson. But what if you want to search for bills on a particular subject? The APIs themselves don’t have much in the way of searching. This is a potential use case for building a local database that can be searched. With a typical database we could store summaries of bills and search for keywords within them. That may be useful if you know what keywords to use. But with LLMs, we have another option.
Embeddings and Vector Databases
To understand natural language, LLMs have to convert text documents into embeddings, large vectors that they can feed into a model. These embeddings are trained to have the semantic meanings of the documents encoded into them. Knowing that, we can use an LLM to translate the documents into an embedding. That makes searching much easier. Instead of trying to match exact keywords, we can translate the query into its own embedding and do a similarity search to find similar documents.
To do this I used the open source database chroma that easily supports these use cases. It can run in memory, persist to a disk, or run in a distributed cloud environment. For this application I’m just going to store the bill summaries retrieved from the government’s bulk data system, so a local database is sufficient. I also used a text encoder model from Gemini.
The first step is to build an embedder to encode the text. All it does is make a call to Gemini to get an embedding either designed for document retrieval (when the text being encoded is a document to be stored) or for a query (when the text being encoded is a query). Then I load the documents and store them in chroma, the database handles generating the embeddings for me. Then I write a query method and write a schema for it so the agent can access it.
With that done, the agent now has access to a powerful search engine.
--> Can you find some bills that might impact the tech industry?
-- calling query_bill_summaries(**{'n': 5, 'query': 'impact on the tech industry'})
Here are some bills that might impact the tech industry:
Advancing Tech Startups Act (HR 5398)
Information and Communication Technology Strategy Act (HR 1354)
Safeguarding Against Fraud, Exploitation, Threats, Extremism, and Consumer Harms Act or the SAFE TECH Act (HR 1231)
Curbing Abuse and Saving Expression In Technology Act or the CASE-IT Act (HR 573)
I can provide more information on any of these bills if you would like.
--> Tell me more about HR 5398
-- calling call_endpoint(**{'endpoint': '/bill/118/hr/5398'})
HR 5398 is the Advancing Tech Startups Act. It requires the Department of Commerce to study and report on the impact of technology startup companies on the U.S. economy. The bill was introduced by Rep. Bill Johnson of Ohio on September 12, 2023. On December 6, 2023, it was ordered to be Reported by the Yeas and Nays: 46 - 0.Using tooling and external data can help you use LLMs to their full potential.