A clean analogy is a library. The Query is your question: “what am I looking for?” The Key is each book’s title or description: “I’m a book about mammals.” The Value is the book’s actual content.
Yes, but in LLMs these are just the letters - I am an "it". Are you relevnat for me? No I am not, "animal" is more relevant.
(1024 dimensions in our case)
What does each dimension of the token tell us? Explain it maybe visually
RMSNorm: keeping the values well behaved
Put this before the first post
Pre-norm
Is this the same as RMSNorm? I would define the difference in those.
Stack 24 of these blocks and the model builds progressively richer representations - early layers pick up syntax, later layers pick up semantics and reasoning patterns.
How do we know which stack tells us what parts of the understanding? It would be great to learn what layers are learning what.
This means solving real problems in ways that are demonstrably better than existing solutions, or even solving problems that otherwise wouldn’t be possible without the new enabling AI technologies. We want to avoid the temptation to implement AI solely for the sake of marketing or competitive parity. Our job is to ensure the perceived value is clear and compelling.
Worth going into detail here about linking the ML success metrics to true product metrics. e.g. Do better recommendations lead to better retention, or orders? There needs to be a translation function set up.
technical debt
Not sure this is a focus, but agreed that infrastructure and budget to power the AI application is key. Do you have easy access to GPUs and scaling of those? How much budget do you have for token generation? How have you optimised the application for input and output tokens, epochs, inference costs etc?
a recommendation is not perfectly aligned with the user’s stated preferences, this can likely be managed in the user experience.
It would be good to explain how. Typically you can't do this - the user experience would not be great and the user would churn if the recommendation was poor, for example
our estimator will have large variance at this point.
Because it has not stabilitised, day 61 we only have 1 day so lots of variance day 1 to 2, then day 3 gets less variance
Fankmata
Factmata
Fackmata
Factmata
Andrea
Andreas
In practice, this often meant that industry would move from rich countries, where labour was expensive, to poor countries, where labour was cheaper. People in the rich countries would either have to accept lower wages to compete, or lose their jobs. But no matter what, the goods they formerly produced would now be imported, and be even cheaper. And the unemployed could get new, higher-skilled jobs (if they got the requisite training). Mainstream economists and politicians upheld the consensus about the merits of globalisation, with little concern that there might be political consequences.
Yes, but isn't this misconception due to immigration laws causing disruption in not just manufacturing products, but services? We get cheaper goods and better products, but is this the case with services? Is a Polish plumber better than a UK plumber?
Journals, which often appeared on cheap, thin paper, were produced almost as an afterthought by scientific societies. The British Chemical Society had a months-long backlog of articles for publication, and relied on cash handouts from the Royal Society to run its printing operations.
Interesting.
Sometimes an investor will ask you to send them your deck and/or executive summary before they decide whether to meet with you. I wouldn't do that. It's a sign they're not really interested.
Interesting to note.
Different plans match different investors. If you're talking to a VC firm that only does series A rounds (though there are few of those left), it would be a waste of time talking about any but your most expensive plan. Whereas if you're talking to an angel who invests $20k at a time and you haven't raised any money yet, you probably want to focus on your least expensive plan.
So important
All they really mean is that their interest in you is a function of other investors' interest in you. I.e. the spectral signature of all mediocre investors.
Love this .
Institutional investors have people in charge of wiring money, but you may have to hunt angels down in person to collect a check.
Interesting.
Some investors will let you email them a business plan, but you can tell from the way their sites are organized that they don't really want startups to approach them directly.
What about LinkedIn? Hasn't this changed things?