Security is another problem, and should be tackled systematically. Artificially making dependency inclusion hard is not it and is detrimental to the more casual use cases.
When you go travelling and do not want to carry around a backpack, and 1 day of heavy video recording, watching youtube on train plus 1 year of lithium battery degradation. That's when I want larger battery.
Performance is worthless if the code isn't correct. It's easier to write correct code reasonably quickly in Python in simple cases (integers don't overflow like in C, don't wrap around like in C#, no absurd implicit conversions like in other scripting languages).
Also you don't need code to be fast a lot of the time. If you just need some number crunching that is occasionally run by a human, taking a whole second is fine. Pretty good replacement for shell scripting too.
But many of the language decisions that make Python so slow don't make code easier to write correctly. Like monkey patching; it is very powerful and can be useful, but it can also create huge maintainability issues, and its existence as a feature hinders making the code faster.
I mean you can see it with your own experience, folks will post a 50 line snippet of ordinary C code in an blog post which looks like you're reading a long dead ancient language littered with macros and then be like "this is a lot to grok here's the equivalent code in Python / Ruby" and it's 3 lines and completely obvious.
Folks on HN are so weird when it comes to why these languages exist and why people keep writing in them. For all their faults and dynamism and GC and lack of static typing in the real world with real devs you get code that is more correct written faster when you use a higher level language. It's Go's raison d'etre.
It isn't an either or choice. The people interested in optimizing the performance are typically different people than those interested in implementing syntactic sugar. It is certainly true that growing the overall codebase risks introducing tensions for some feature sets but that is just a consideration you take when diligently adding to the language.
Because many never used Smalltalk, Common Lisp, Self, Dylan,... so they think CPython is the only way there is, plus they already have their computer resources wasted by tons of Electron apps anyway, that they hardly question CPython's performance, or lack thereof.
I use Python 90% of my day and I can't say I like it or hate it or care about it at all. I use it because it has all the libraries I need, and LLMs seem to know it pretty well too. It's a great language for people that don't actually care about programming languages and just want to get stuff done.
Most of the time you are waiting on a human or at least something other than the cpu. Most of the time more time is spent by the programmer writing the code than all the users combined waiting for the program to run.
between those two, most often performance is just fine to trade off.
Because computers are more than 100x faster than they were when I started programming, and they were already fast enough back then? (And meanwhile my coding ability isn't any better, if anything it's worse)
I don't think anyone aware of this thinks it's a good tradeoff.
The more interesting question is why the tradeoff was made in the first place.
The answer is, it's relatively easy for us to see and understand the impact of these design decisions because we've been able to see their outcomes over the last 20+ years of Python. Hindsight is 20/20.
Remember that Python was released in 1991, before even Java. What we knew about programming back then vs what we know now is very different.
Oh and also, these tradeoffs are very hard to make in general. A design decision that you may think is irrelevant at the time may in fact end up being crucial to performance later on, but by that point the design is set in stone due to backwards compatibility.
It isn't. There are many things Python isn't up to the task. However, it has been around forever, and some influential niche verticals like cyber security Python was as or more useful than native tooling, and works on multiple platforms.
Law should be considered to be artificial rules optimized for the collective good of society.
What's the worst that can happen if we allow unregulated AI training on existing music? Musician as a job won't exist anymore lest for the greatest artists. But it makes creating music much more accessible to billions of people. Are they good music? Let the market decide. And people still make music because the creative process is enjoyable.
The animus towards AI generated music deeply stems from job security. I work in software and I see it is more likely that AI can be eventually able to replace software devs. I may lose my job if that happens. But I don't care. Find another career. Humanity needs to progress instead of stagnating for the sake of a few interest groups.
I don't work as a musician so it's nothing to do with job security -- I think that using artists' output without their consent in order to train a soulless AI model for some tech middleman to profit from is repugnant, and the cheap rhetoric about democratizing music and "bringing music to the masses!" adds insult to injury. I can guarantee if OP's intellectual property was violated in this project, like somebody ripping off their model or trademark, they'd be suing, but they conveniently handwave away mass scale IP theft when it comes to musicians.
Andrej's video is great but the explanation on the RL part is a bit vague to me. How exactly do we train on the right answers? Do we collect the reasoning traces and train on them like supervised learning or do we compute some scores and use them as a loss function ? Isn't the reward then very sparse? What if LLMs can't generate any right answers cause the problems are too hard?
Also how can the training of LLMs be parallelized when updating parameters are sequential? Sure we can train on several samples simultaneously, but the parameter updates are with respect to the first step.
As I understood that part, in RL for LLMs you take questions for which the model already sometimes emits correct answers, and then repeatedly infer while reinforcing the activations the model made during correct responses, which lets it evolve its own ways of more reliably reaching the right answer.
(Hence the analogy to training AlphaGo, wherein you take a model that sometimes wins games, and then play a bunch of games while reinforcing the cases where it won, so that it evolves its own ways of winning more often.)
AlphaGo seems more like an automated process to me because you can start from nothing except the algorithm and the rules. Since a Go game only has 2 outcomes most of the time, and the model can play with itself, it is guaranteed to learn something during self-play.
In the LLM case you have to have an already capable model to do RL. Also I feel like the problem selection part is important to make sure it's not too hard. So there's still much labor involved.
Yes, IIUC those points are correct - you need relatively capable models, and well-crafted questions. The comparison with AlphaGo is that the processes are analogous, not identical - the key point being that in both cases the model is choosing its own path towards a goal, not just imitating the path that a human labeler took.
In my experience Linux can have some driver bugs on specific hardware that windows doesn't, like not waking up after suspend on some Nvidia cards with some drivers, etc. But it handles hardware issues miles better.
90% of hard drives that windows does not detect Linux can detect and copy 99% of the data with some IO errors for the rest. Can handle hardware instability like bad rams or too high of an overclock for ages while windows crashes very easily.
It's questionable a mere algorithm would get us there without a fundamental change in computer architecture. (in terms of Intelligence / W)