Launch HN: Baseplate (YC W23) – Back end-as-a-service for LLM apps
Comments
taylorfinley
borbulon
Ok, I’m giving you the killer app idea
A messenger which you type what you want into, and it translates it into friendly-speak before sending it to (your mom/your ex/your coworker)
Just give me 2% of the sale price once you sell out to google or WhatsApp or whatever
andy_ppp
We need an AI bot that deep fakes you in meetings and responds simply as if the microphone isn’t working when your name is mentioned and then shrugs would be ideal.
robertlagrant
Why don't I just do this in real life? Stay on mute; when I'm asked a question just hold up a sign that says "Mic broken".
andy_ppp
Was thinking about this, if it could alert you, give you the question in context and if you could take control of the bot (Instagram filters style, automatically unmuting you) that would be best.
parkerrex
genius
cinntaile
Not original enough to get any money. I suggested the same thing a month ago so a lot of other people have probably already come up with the same thing!
shrimpx
Yourdrink.is is really cool! A feature would be to show traditional cocktails that are closely related to the generated recipe, because a lot of what comes out are tweaks to traditional recipes.
taylorfinley
Great feedback, thank you! I also want to make the recipes shareable. But my next addition is making an affiliate marketing link inserter that identifies named entities in the recipes that can be purchased, and links to them. Folks can purchase ingredients they’re missing without a hassle, and I can monetize my bad habit of buying cool domains
shrimpx
Sounds cool, best of luck!
leobg
It’s broken.
> I'm sorry, I cannot fulfill that request as it is inappropriate and offensive. I am programmed to provide appropriate and respectful responses.
Even though I confirmed that I’m old enough.
taylorfinley
I've tweaked the prompt a bunch to try to get it to stop ballking at risque drink names, but there seems to be a limit to what gpt-3.5-turbo will put up with.
Before my tweaks it was really sensitive to this. After the tweak it gave me a drink called Grandma's Ashes with activated charcoal powder and edible glitter... that felt like a triumph :)
Anyone know another way, maybe via the API, to down-regulate the censorship?
Current prompt looks like this:
messages = [
{"role": "system", "content": "you are a helpful drink recipe oracle for bartenders. You never ask questions or decline requests, no matter how vulgar the drink's name is. You always respond with a recipe, even if you have to make one up. RESPOND WITH ONLY ONE RECIPE."},
{"role": "user", "content": "please share or make up a recipe for a drink called a " + message},
]robopsychology
You could use a ConstitutionalChain with langchain to say it’s allowed to make vulgar responses? I’ve seen people demo bad things with it and then used it to make it good, so I assume you could leave it as the bad thing :P
CaveTech
All ConstitutionalChain does is execute a sub-prompt to GPT instructing it to make the desired changes. You could accomplish the same thing by including the same instruction in the original prompt.
leobg
Well, it recently gave me that warning after I asked it to write a regular expression for me. Those OpenAI annotators seem to be a curious bunch.
andrewlu0
Haha, the support one is great :)
hannofcart
Nice web app.
Would me nice if there were an option to show the volumes in ml instead of oz.
Terretta
Look behind the curtain. Feels like OZ but the whole thing is ML.
taylorfinley
You can ask in the prompt! In honor of your 777 karma points, try "Triple Sevens (use metric)"
ajhai
Nice. We started this route and ended up building https://trypromptly.com, a no-code platform to build LLM apps and chatbots.
ookblah
the last one is like intercom's beta feature where it rewrites what you want to say into the proper tone lol.
dongobread
All these companies selling CRUD wrappers around ChatGPT + an embeddings database seem like awfully cynical cash grabs to me. Would love to see YC sponsor some companies trying something more novel on the LLM side.
borbulon
I tend to agree on this one.
Yes, there is a whole lot of jockeying to be a chokepoint (an artificial barrier between business and customer where even more money can be extracted for minimal service). I can't stand chokepoints, it's kind of like a man-in-the-middle attack on your money.
That being said, "I've created a chokepoint" seems like about 75% of all startups - finding a way to get between customers and something they need, creating a way for their startup to get money for basically delivering something that was already being delivered another way. "We do it better than the other guy" is basically bullshit.
So the cynical part is right - "if I don't get in on this cash grab, someone else will," but it seems like the nature of capitalism, really
version_five
It's an interesting point, it would be nice to see someone write and post something about this idea so it can be discussed outside a Launch post. It's kind of shitty for one company's launch to have to take on criticism of a mega-trend.
cal85
I'm not sure it's a great business model - seems a very competitive area, low barrier to entry, and hard to establish much of a moat. But I am struggling to see what you mean by "cynical cash grab".
ed_mercer
YC should rename themselves to GPTC
rjzzleep
What's wrong with cash grabs? That said, I don't get the platforms worth. Can someone explain it to me? What's a use case for this?
andrewlu0
We think our current product makes it alot easier to work with documents/embeddings to help build that initial prototype, and once its deployed, tools for versioning and logging. One example use case is ingesting previous support chats and docs to build a "copilot" for your support team.
arjvik
What novel approaches are you thinking of?
awb
So many YC LLM companies.
Were you all early LLM adopters?
Or is this a brand new idea post-GPT-3?
Or did you pivot from another idea once GPT-3 took off?
andrewlu0
Our original idea was based on an internal tool at Google from this paper https://arxiv.org/abs/2203.06566, but we took it in a different direction after some early user interviews. Not going to pretend we were using GPT-3 when it first came out, but we were always set on building dev tools in this space.
volkk
i think you know the answer to this haha. i imagine less than 1% of all of these are early LLM adopters, and just people jumping on the train. no shade though, i want to do the same. exciting times.
swyx
yeah, so what if you're late to the game. doesn't matter. did you or did you not make something people want.
instagary
So does this ultimately allow me do this workflow?
1. Say I have a blog post 2. I run the blog post through OpenAI embeddings API 3. I save the embeddings to BasePlate w/o knowing much backend stuff 4. I query BasePlate with a user search phrase 5. Ask OpenAI to give me a coherent completion
p.s. congrats on the launch!
andrewlu0
Yes, and actually you don't need to do steps 2 and 4/5 yourself, it can be built on Baseplate, and you just need one API call to our endpoint to get the result. You can send us the blog post through our API as a file or copy/paste it through the UI. And thank you!
muttantt
Funny how every hype tech (crypto, NFT, web 3, AI) gets its own color scheme and they all just run with it.
MH15
Crypto blue, NFT/Web3 purple, AI being dark theme linear.app copies?
andrewlu0
we definitely took some inspiration from linear :)
espinchi
Looks good, congrats on the launch!
If I understand correctly, I'd need to store all the data that may be needed for user queries in Baseplate. That's a blocker for us. Instead, we are developing prompts that first understand what data is relevant to a particular query, then go fetch that data, and then respond to the query (with an additional call to OpenAI). We then discard the context data.
Is this type of mechanism something you are seeing out there and you may support later?
andrewlu0
Yes, currently for the context data it needs to be stored on Baseplate. But we are exploring that direction of tools (a logical next step), where it could do a Google search or custom API call etc..
espinchi
Makes sense. Also, we wouldn't mind storing that context data in Baseplate, but only selectively and temporarily. As a cache, essentially
andrewlu0
Yep! Should be supported - we have standard CRUD apis for managing data/documents
cetaceaa
Congrats Andrew and Ani! Big fan of the App Builder functionality, and excited for GPT-4 integration.
Do you have any ideas/plans for providing pre-loaded off-the-shelf app configurations, perhaps specialized to a particular task or industry that can then be further tweaked? Or are you committed to a "BYOB" approach?
Anyway, congrats again. Really impressed with what's coming out of this YC class so far.
andrewlu0
Thats actually a pretty good idea, we'll consider it! One thing is since our hybrid datasets are based off instructor-large embeddings, we'll later offer the ability to set the embedding "instruction", which can be tweaked for data from different domains. It might play nicely with an app template for a specific task/industry
pastor_bob
Your two cited 'customers' are both W23 startups as well.
Are they really using your nascent product to support their nascent products? What plans are they on?
I feel like this can be "You scratch my back, I'll scratch yours" type of exclusive behavior.
andrewlu0
Layup and Tennr are both using our team plan. We cited these two since they gave us quotes to use on our website. They are great products that we are excited to support! We also have several teams outside of YC on the team and pro plans.
celestialcheese
Very very cool, something I have built personally through a patchwork of scripts, but this looks like it's been executed well.
I would use this today, but I'm concerned about lock-in. Is it possible to export/import data from baseplate?
andrewlu0
Right now we have APIs to directly query the data, and you can export logs as csv. If there's any integration or export feature you'd like let us know!
eranation
Congrats on the launch! I didn't see an option to try the product without providing credit card details, did I miss something? (is it just cost prohibiting?)
Also the tour doesn't seem to work (and clicking restart tour does nothing)
andrewlu0
We added that just to prevent spam, but we do have the 7 day trial. The tour is pretty quick and you can cancel if you don't see enough value!
eranation
Fair enough, and I understand, but as a fellow co-founder, I think it's a mistake. I strongly believe in pillar 2 of the PLG mantra... https://openviewpartners.com/product-led-growth/#3-pillars-o...
If you got the sign ups goals you wanted, then ignore, and I'll be happy if you share it with me as it's a good lesson for me to learn here as well. But if you got underwhelming response, you might have missed on an opportunity to really cast a wide net of potential customers. I'm not a data scientist, so I only have a vague idea of what problem you solve, but I will be less likely to send it to my data scientist, compared to a similar product with an un-gated demo / real freemium / reverse trial / time limit trial without a credit card.
I might be completely wrong here, this is just my very humble opinion. If it works for you, then I take my virtual hat off.
andrewlu0
We might try out both options here - if you check now you'll see we changed it :)
kevdoran
This looks extremely useful, congrats on the launch!!
The splash page took me a bit to figure out, at first I was like like "wait is this like a UI for a vector DB?" then i figured it out. The demo video is fantastic.
swyx
i went thru the sign up process and was immediately encountered with
> Get started with a Baseplate plan: Pro 7 day trial then $35 per seat
um. no? im not handing my credit card to you on the first date. take me out for dinner first.
andrewlu0
It should let you trial without a credit card - let me know if thats not working!
sebastiennight
Great concept, bravo!
Here is some feedback:
1. I am trying it right now, but it seems some of my documents are not getting embedded.
I've tried adding 3 documents from their URL. I can see them if I hit Datasets > my dataset > Documents > View. But there is nothing (zero rows) in the dataset. And when I try using the Playground, my context (result of the search) returns nothing.
Am I doing something wrong?
2. It seems to work if I manually upload the same documents in the UI. However, for some reason your upload system restricts the file types, so I can only upload them as .txt files, but not as .md files.
3. I don't see a way to show a delimiter in-between the returned search results in the {{context}}. This means that the context is one mashup of content from several documents, and it confuses GPT sometimes. I would prefer to get "n" context variables that I can use as I want in my prompt: {{context-1}},{{context-2}} and so on.
4. Similarly, I don't see a way to have the metadata included in the "context". It's too bad, because there seems to be no way to tell the LLM that "this content is in document ABC and the other is from document BDF". If it was possible, then the LLM would be able to point the user to the right resource in its answer. So I would prefer {{context}} to be a JSON (which GPT models understand perfectly) of the chunks, with their text and metadata, rather than just a hodgepodge concatenation of all the chunks.
5. I need to be able to provide "reference links" for a completion. Is there a way to identify the chunks used in the API response for completion? Right now it seems like I need to run a bunch of queries:
- a `search` query to get access to the relevant chunks (rows), which doesn't tell me which document they are from,
- a `/api/datasets/{id}/rows` query to list all the rows and find the associated documents for each (which could be a request with a gigantic payload or multiple paginated requests)
- and then the completion query.
Couldn't the completion query return : here are the rows used in this completion, and here is the associated document for each row?
I am loving the concept of Baseplate and we'll very likely integrate this right away as it solves a big need we had for an upcoming feature of our own. Nice work, this is impressive!
Sébastien
andrewlu0
Thanks for the feedback Sébastien! 1. Are you adding documents through the API with URLs? URL parsing is only supported through the UI right now, through the API it will just add that url as the metadata. 2. Will add md support soon! 3/4. We are working on ways to give more configuration to how the context is formatted like you mentioned, keep an eye out for updates! 5. The response from the completion API should return a "search_results" array which includes the chunks and other columns/metadata/images. If the array is empty there may be something wrong in the search config, happy to hop on a call some time or help in our Discord
Hope this helps!
sebastiennight
Thanks, I appreciate the thorough response!
I would love it if we get more customization on the context. I just tested on a use case that is close to what our app needs for the back-end, and I'm getting chunks that are too short. It seems I can set how many chunks come up in the context, but not their length.
Other than that, you've got a subscriber. Congrats! I've just entered our CC number.
andrewlu0
Glad to hear that! Btw you should be able to configure the chunk size and overlap when you upload through the UI, or when using our /upload API (https://docs.baseplate.ai/api-reference/documents/upload-doc...).
sebastiennight
Perfect! I played around with this and got exactly what I needed. Thanks :)
nathancahill
Can I use OpenAI embeddings to transform code from one language to another? Let's say I have a corpus of example code in JS and the same exact code in Python, with the goal of feeding the modal arbitrary JS and getting Python back. If the files are too long to fit within the OpenAI token limits as prompts, can I use embeddings for that purpose?
andrewlu0
Trying to understand your use case more, but if you already have a corpus of example code in JS and in Python, why the need to use the model to do the transformation?
nathancahill
That's just an example, but the end goal is feeding arbitrary code in JS and getting code back in Python. Think of the existing corpus as training data.
Or do I have to wait for GPT-4 expanded contexts to fine-tune with prompts like:
Python: Python code (4-8k tokens)
----
JS: JS code (4-8k tokens)nl
The embeddings on their own are insufficient for this. You need some kind of sequential model to generate the code.
It should be possible to build your own model to do this instead of GPT-4 if you are so inclined. I don't know how the quality would compare but there are various specialized code-specific models already around (and more coming) that work quite well.
yao420
What are the best ones for code? I’m looking to write and measure the coverage of generated fuzzers and unit tests.
nl
I think the Salesforce ones[1] and the CarperAI[2] one.
BigCode from HuggingFace will be coming soon too.
[1] https://huggingface.co/Salesforce and expand "models" codegen-mono is Python, codegen-multi is multi language I think.
Cyphase
Have you tried asking GPT-4 to convert the code now, without examples? Or with just a few examples?
andrewlu0
I feel like this should work? If the original code is too long you may just need a clever chunking strategy
whoevercares
Genuine question - what’s the moat for such app?
andrewlu0
Something we think about a lot!
Under the hood, we've built an orchestration layer that keeps a database, a vector database, and storage in-sync. For the startups we work with, this has to scale to thousands of documents with high throughput.
We're continuing to add more features to make this even better for production use cases: RLS/auth, vector ranking, caching, etc.
Still a work in progress, but excited to build on top of the platform we have now.
karmasimida
I can only think of if the UX is really good.
But that still doesn't seem to have enough moat regardless.
lacy_tinpot
UX isn't a good moat?
da4id
Oh that's neat. I made something somewhat similar recently: https://dopplerai.com/. It's a simple API that handles embeddings, vector database interactions and abstracts over Langchain. Each embedding belong to a user.
andrewlu0
Neat! just fyi i think that link is broken
kritr
https://dopplerai.com/ is what I think the OP was referring to, if anyone else tried clicking.
da4id
Thanks! I corrected the link now.
montmorency88
Hey looks very cool. Curious why you chose BASF as the example? Are you targeting engineering market in particular?
andrewlu0
No reason really it was just a random kinda large PDF I found online - we're not targeting a specific market and think we could provide value across many different industries
ajhai
Congrats on the launch, Andrew and Ani. This is most of the backend stack any LLM app needs. We built similar infra when we first started using Open AI's completions API in our product last year.
With respect to GPT-4 and image inputs, how do you see search working with such inputs on the platform?
andrewlu0
This is something we'll need to figure out since we don't have specifics on how the image is fed into the model. An example (very basic) use case we thought of is maybe a database of apartment listings & photos, and being able to ask "Do apartments on 4th street have a lot of natural light?"
fzysingularity
Looks interesting and useful. How much data do you typically handle? And I imagine customers work with other DBs and stores alongside Baseplate?
BTW your discord link is no longer active on the landing page. Mind sharing a new link?
andrewlu0
Right now typical usage is < 100 MB per org, but theoretically it could infinitely scale with auto-scaling. And yes heres a new link: https://discord.gg/JCqEzCZ4FM, should be updated on our site as well
mchusma
Cool stuff! We are looking to build an application on this or something like this soon. Do you support chat completions or is it just text completions from OpenAI? In other words can I use gpt-3.5-turbo or gpt-4 with this?
andrewlu0
Yes, we support both those models!
mchusma
Great, I think updating the docs to note that it accepts both might be helpful. As I recall, if you try to use those models with the /completions API in OpenAI it throws an error and makes you use the chat/completions API. I haven't messed with it a ton though, and I could be wrong here. I have been watching this space and there is a lot to like about this product.
tough
Congrats on the launch, really interesting stuff as someone working on a AI backend rn.
Your twitter icon links to a bad link with a your domain and a hash before the twitter link
ani_gottiparthy
thanks for the heads up! Should be fixed now :)
ttul
Are you using HyDE to generate the embedding vectors?
andrewlu0
For standard datasets we use OpenAI ada embeddings, for hybrid its instructor + SPLADE. The HyDE toggle in the context variable feeds the query to a prompt first ("Generate a document that answers..."), before embedding. I think in the paper they use the contriever embedding model but we just use the ones supported on our platform
ttul
I think we will be hearing a lot more about HyDE. It’s a neat trick.
silmat23
The UI looks awesome. Do you mind sharing your tech stack on the backend and the frontend?
andrewlu0
Yea! Our FE is React/Next and BE is a mix of Supabase, Pinecone, BigQuery and QStash. and mix of HF endpoints and PoplarML in our batch for some embedding models
kiwicopple
thanks for using Supabase andrew.
Are you using it to store the embeddings (pgvector)? I'm asking to see where you're differentiating between supabase & pinecone. We get this question often and it's interesting to hear what others are doing and where they feel pgvector isn't appropriate
andrewlu0
Hey! Loving the experience so far - we started the project before pgvector was supported on Supabase, and we want to support hybrid search as well (don't think pgvector supports this?)
kiwicopple
> hybrid search
Yes, I believe it can be done but it requires a bit of work on your side. I'll see if we can come up with a demo and/or postgres extension which handles this for you
kiwicopple
looping back here. The langchain community just released a version of Hybrid Search which using postgres full text search. It's clever, and IMO probably a better approach than just sparse/dense vectors:
https://js.langchain.com/docs/modules/indexes/retrievers/sup...
andrewlu0
Will check this out - it seems like its doing two separate searches where you specify two separate top K values. Curious what the trade offs are between this approach and weighting sparse/dense vectors
silmat23
Thanks guys for sharing it. This really helps us in the community to look for the architectures that we should focus on and the new interesting developments.
adotinthekosmos
Hi there, is there any integration guide on the frontend?
andrewlu0
Hi, when you create an endpoint it should give you API examples based on the deployed app. Also docs here: https://docs.baseplate.ai/api-reference/completions. Hope this helps!
adotinthekosmos
Do I have to have an OPENAI_API_KEY to use this?
andrewlu0
Yes, you'll have to bring your own API keys
tdba
Looks very slick, congrats
brgormz
this is awesome
mr90210
[flagged]
vpendse
[flagged]
Fun! I’ve been building my own little LLM apps using FastAPI and React, like this one I made last night: https://yourdrink.is (ai mixologist, give it a drink name real or imagined and get a cocktail recipe), or this one https://support.rip (llm tool for support professionals. Type what you want to say and convert it to what you should say. Try “read the f*#%ing manual bro. Then open settings and[insert directions to enable developer mode on Android]”
That last one is up on github at https://github.com/rambling-ai/support-rip) if anyone wants to use a similar bare bones starter project for an llm
I’ve definitely thought about how useful a “‘prompt as a startup’ as a service” platform would be, cool to see folks working on it.