More room for your work
Work toward AI that can use more of a document or conversation at once.
Long context.
Closer to home.
Our goal: AI that can work with more of your documents on a graphics card you can own.
More room for your documents. More control over where you run AI. We are researching how to make both practical on hardware people can own.
Research in progress. Measurements below are preliminary.
01 / What we are building
Useful AI on hardware you can own.
Research toward AI that can work with longer documents without needing specialist hardware.
Make limited memory go further
Study how to run AI within the memory available on consumer graphics cards.
Make the limits clear
Build experiences that explain what has been tested and what still needs work.
02 / Why it matters
Keep the possibilities close.
Use hardware you can own
Make more capable AI practical without relying on specialist computing equipment.
Keep more of the story together
Give documents and conversations more room, while testing whether answers remain useful.
Choose where your text goes
Running AI on your own computer can give you more control. This is a research goal, not a privacy promise for a future service.
03 / How it works
Make more room for what matters.
Longer documents need more computer memory. We study ways to use that memory carefully, then test speed and answer quality separately.
Keep a compact record
Explore ways for AI to keep track of what it has read using less memory.
Find the useful detail
Study how AI can return to relevant information as a document or conversation grows.
Check the results
Measure how fast the system runs and test the answers it gives. Speed alone cannot tell us whether an answer is good.
04 / Preliminary research
Measured, with the limits in view.
Existing tests across graphics cards with 8, 12, 16 and 24 GB of memory. Each figure below uses the same long-text workload.
8 GB / RTX 3060 Ti
49.71 tokens/s
6,118 MiB of graphics memory used by the test process with 131,072 text tokens already in context.
Preliminary research measurement ·
12 GB / RTX 3060
39.98 tokens/s
6,084 MiB of graphics memory used by the test process with 131,072 text tokens already in context.
Preliminary research measurement ·
12 GB / RTX 4070 Ti
82.89 tokens/s
6,128 MiB of graphics memory used by the test process with 131,072 text tokens already in context. This card was tested locally; the others shown were rented.
Preliminary research measurement ·
16 GB / RTX 4080
104.02 tokens/s
6,231 MiB of graphics memory used by the test process with 131,072 text tokens already in context.
Preliminary research measurement ·
24 GB / RTX 4090
140.01 tokens/s
6,397 MiB of graphics memory used by the test process with 131,072 text tokens already in context.
Preliminary research measurement ·
These tests used random model weights: they measure running speed and memory use, not useful answers. Tokens are pieces of text. Speed here counts new tokens after the input has been processed; it is not the time from asking a question to receiving an answer. Each test ran one conversation. Results describe these test setups, not a promise for your computer. There is no released model or public AI service here.
Read what was tested ↗05 / Roadmap
Each step has to earn the next.
This is the path we are working toward, with no promised release date. Each step depends on the results of the one before it.
Develop and test
Build a research model and check its answers against tests chosen in advance.
Test on everyday hardware
Check speed, memory use and correct operation on consumer graphics cards.
Try complete tasks
Test the experience from question to answer under defined conditions.
Consider wider access
Decide what can be offered after the evidence and release review support it.
06 / Questions
A few things to know.
What is BDHM working toward?
Our goal: AI that can work with more of your documents on a graphics card you can own. We are researching ways to use limited computer memory more efficiently. This is work in progress, not a released product.
What does long context mean?
It means the amount of material an AI can work with at once, such as a document or conversation. More room for text does not, on its own, mean more accurate answers.
What do the research figures tell me?
They show how fast a test system produced text and how much graphics-card memory it used in specific tests. They do not measure answer quality or promise the same results on your computer. Read what was tested and its limits.
Can I try a model here?
There is no model to try or account to create on this website. Contact us to discuss the research.
Does this website track me?
The site has no analytics, cookies or forms. Images and fonts come from this site. The theme button remembers only your light or dark preference on this device. Hosting may keep connection logs. Read the privacy page.
What is BDHM-Labs?
BDHM-Labs is an independent research lab working to bring useful AI to everyday computers.
07 / About
Independent research. An accessible goal.
BDHM-Labs is an independent research lab working to bring useful AI to everyday computers.
We study how AI models and the software that runs them can make better use of limited memory, so more people can use them.
08 / Contact
Start a conversation.
Interested in the research? We welcome questions and conversations.
Contact us ↗