I'm extremely amused that one of the trials here was "how do I measure 6 liters of water with a 12 liter jug and a 6 liter jug?" and the article completely glosses over the fact that the framework doesn't help GPT find the simple answer "fill the six liter jug"
The whole “prompt engineering “ space has a smell of secret sauce that is ultimately not IP or even particularly clever. Reading the SOTA systems prompts and other prompt optimisations leads me to believe that any business model based in prompt manipulation is ultimately offering a zero marginal value product, not a
Zero marginal cost one.
We call those "Snake Oil Mines" in the military industrial complex. They're great! Legendary examples include the Norden Bombsight, MBSE, AS9100 Consultancy[1], and Six Sigma. "Unfalsifiable results" means "you need more".
The beauty is you can make your very own snake oil based on them. All kinds of clever optimizations when no one's sure what the end result is supposed to look like.
EDIT I'm sorry if it seems like I'm picking on "AI" stuff. I'm really not. I'm picking on people buying products without enunciating what it is they're trying to do.
[1] When it's not just straight extortion, i.e., the auditor "selling" consultancy services to make sure you "pass"
I am always amazed at these highly engineered prompts where the negative space is full of terms like “grotesque”, “deformed”, “extra toes”, etc…. It feels so much like waving a dead chicken over it. A very superstitious practice.
yeah. How many images are in the training dataset labelled with "extra toes" guys? How about "malformed limbs"? Lets at least have some understanding of how these things are trained and work if we're gonna be selling services based on them.
Grotesque might conceivably have some effect. but its pretty clearly cargo culting
Yeah I get the same sense. Prompt engineering is short term need while models haven’t really been productized very well. I would certainly expect to be less of a need in the coming couple of years when LLMs are refined and targeted to specific use cases.
Basically the only reason we need prompt engineering is that current models still aren’t that good (relatively). If they were, they would be able to help themselves with the silly task of prompt engineering without all the superstition.
The best way to always know the prompt is to write it yourself. 90% of what those libraries do is just a complex front end to a templating system. So you are often better off just using f-strings or template literals.
> Furthermore, the prompt has a spelling error (Let'w) and also overly focuses on the negative about identifying errors - which makes me skeptical that this prompt has been optimized or tested.
I want some kind of genetic algorithm where I can choose between perhaps three images/renders and then have the AI iterate on the one I chose to make three more variations. Perhaps the software varies my prompt for me in clever ways?
Otherwise I have no idea how to take my prompt and a disappointing outcome and massage it.
I wrote some code to annotate the image with the text of the prompt. I couldn't find a convenient way to do it in EXIF.
For Midjourney: https://github.com/ernop/social-ai/tree/main
This one just downloads and annotates all the images in a discord channel/server you have admin in, and can backfill.
We felt this 100% so we built a DSL (called BAML) to solve the "prompt transparency" problem (amongst other issues). We have a VSCode playground that always shows you the full prompt -- kinda like a markdown preview works.
>Some of these frameworks like instructor use like 80% more tokens or only work with OpenAI so we aim to tackle all these problems from the ground up.
No doubt this is because open source OpenAI API implementations don't support function calling, but now that we have context free grammars in transformers (and TGI) implementation of function calling is not very far off.
Also, about the article, I chuckled, because he listed "guidance". That is (other than an api) a feature of some AI frameworks to implement negative prompting. For example in Transformers. You can send a prompt "what is a tomato", alongside a guidance prompt "you're very verbose" and a (negative) 2 guidance scale. This way the answer will be the opposite of "you're very verbose". This feature is not in the official release yet, but it exists (in git) and it is very neat.
Not sure it's related to function calling. GPT4 can do function calling without using the specific function-calling API just by injecting the schema you want into the prompt with directions and asking it to return JSON. It works like >99% of the time. Same with 3.5-turbo.
The problem is these libraries convert pydantic models into json schemas and inject them into the prompt, which uses up like 80% more tokens than just describing the schema using typescript type syntax for example. See https://microsoft.github.io/TypeChat/, where they prompt using typescript type descriptions to get json data from LLMs. It's similar to what we built but with more boilerplate.
> If the title contains a gratuitous number or number + adjective, we'd appreciate it if you'd crop it. E.g. translate "10 Ways To Do X" to "How To Do X," and "14 Amazing Ys" to "Ys." Exception: when the number is meaningful, e.g. "The 5 Platonic Solids." Otherwise please use the original title, unless it is misleading or linkbait; don't editorialize.
The post title is "Fuck You, Show Me The Prompt", but the HN title is "Show Me The Prompt". The removal of "Fuck You, " has significantly changed the title IMO, and is a form of editorialization. Did the HN mods do this, or was it submitted in this form?
I changed it because "Fuck you, show me the prompt" is baity. We don't have a problem with profanity (https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...) - I'm sure we've restored more "fuck"s in titles over the years than we've removed. We have a problem with linkbait, though, as the guideline you quoted indicates.
Btw, there's another issue. We put the OP in the second-chance pool (explained at https://news.ycombinator.com/item?id=26998308), meaning it would get a guaranteed placement on HN's front page. That's already such a steroid shot that the post can do without a jacked title.
imho, just to let you know, I think the original title, more than being baity, it clearly describes a growing sentiment when using the never ending list of AI tools. The prompt is important and not being able to see it, we cant tell if could be improved, and can't test the same prompt on different models. My perception is that the original title is more acurrate. Anyway, thanks!
Nope, that was a bad decision IMO. As "kingforaday" points out (in https://news.ycombinator.com/item?id=39391466), you've -- self-referential irony, in this context? -- fucked with the headline that would prompt more or less artificial intelligences to click on the post.
Hi Jeremy, Hamel here. I submitted this blog post, and definitely did not submit it in this form. One of the defining features of this blog post was the F word.
Since the article discusses interception and rewriting prompts, I find this entertaining. If your statement is true, hopefully others will enjoy the irony as I did.
To clarify -- I don't mean this as a criticism. If HN wants to censor swearing, that's fine; but it probably should be mentioned in the guidelines in that case.
