This story plays out so often that here should be a law about it: Supply lags demand, prices soar, everybody hears about it, everybody pours in, supply surges, demand normalizes, supply overshoots demand, prices collapse. Already happened to software engineering, data science. Keeps happening to hardware production every few years. Sounds like AI research is headed that way too.
"The boom and bust cycle is a process of economic expansion and contraction that occurs repeatedly. The boom and bust cycle is a key characteristic of capitalist economies and is sometimes synonymous with the business cycle." [1] Somewhat similar is the bullwhip effect[2], although there isnt a long supply chain for labor.
This one’s a bit worse than meta’s usual sins: enabling political ads to manipulate the user is one thing. But enabling naughty people to generate naughty pictures of innocent bystanders because they happened to be in the field of view of an idiot talking to their glasses is a whole different level. Would be surprised if this is legal.
> CV Raman won the Nobel prize in science, Tagore won the Nobel prize in literature, Ramanujan etc, the names are numerous.
So golden age of India was when the country with a seventh of the world's population won 2 nobel prizes over 5 decades ?
> The British rule also was the largest and most stable unification of India till the modern times. After 1850s there were almost no pockets of military resistance against the British rule.
Mughal and Gupta empires lasted over 3 centuries, Mauryan empire a little under 1.5 centuries. By comparison, east india company rule lasted a century and the British crown's rule less than that. So again completely incorrect.
> The third golden age which no one wants to admit (left or right) is the British Golden age.
There's your hint: if people on both sides of the aisle don't "want to admit" something, maybe it doesn't make sense. Not to mention a slap in the face of billions of Indians.
> The British age declined with WW1 and WW2, and ended with Indian independence.
Thank god for that decline, otherwise Indian taxpayers would have been funding Brexit and the crumbling British economy right now.
> My oversimplified summary has been
This is not a summary, it's a lazy opinion backed by little research.
Have you read any books at all by Indians who lived through the British empire. Maybe start with “My Experiments with Truth” by Gandhi. The caricature that some modern Indians have made of the British empire would make even Gandhi turn in his grave.
But if you want to read something really heretic, maybe try reading An autobiography of an unknown Indian by Niraj Choudhary. Choudhary was a British raj supporter, as in an Indian who opposed Indian independence. Does that shock you? There were actually quite a lot of them, more than you’d expect.
Then if you want to get really metal, read in his own words, by Subedar Sitaram Pande. Sitaram Pande, was a soldier for the bengal army, for the British empire from 1812 to 1860, it’s one of the rare first author accounts we get of an Indian in that era. It will give you a glimpse of how an Indian at that time thought generally (hint: it was far more dominated by caste than you’d expect), how he viewed the empire and his relation to it. At that point I would say you are ready to try to understand Indian history that is not an avengers movie plot.
I would follow it with CK Majumdars history of modern India, one of the best historians so committed to the truth that Nehru had to throw him out of the government and try to prevent him from writing his book. Don’t worry, he’s not an heretic, he was an Indian freedom fighter, but you will find that he was far more honest about his life under Britain, under Indian national Congress and the state of the country in different periods of time (he also has a 12 volume set covering India for over 2000 years that I never had a chance to complete).
Please don't cross into personal attack or flamewar. Your post here is a noticeable step in those directions, and we're trying for the opposite on this site. (I'm not saying that the parent comments were perfect either but degrees matter.)
>Mauryan empire a little under 1.5 centuries. By comparison, east india company rule lasted a century and the British crown's rule less than that
This is a very dishonest way to obscure the actual facts.
Direct rule from Britain lasted for almost 90 years: 1858 to 1947. Even by your numbers then, that's 190 years: longer than the Mauryan empire's whole lifespan, and much closer to that of the Mughals. From there the question remains whether it's the longest "unification", and this mostly comes down to exactly when each of the aforementioned empires could be considered to have "unified" India.
By any definition the Mughals united the subcontinent by 1707AD at the latest: but by 1751, less than fifty years on, their effective domain had declined to a few pockets in Rajputana and Bengal.
The Guptan Empire on the other hand, while certainly a key predecessor to later Indian states and a major unifying force in the northern half of the subcontinent, never conquered the southern half -- what is today Karnakata, Kerala, and Tamil Nadu never entered their control. The closest they got was ~420AD after the south-eastern conquests of Chandragupta II, but again within fifty years they again lost control of today's Orissa, and even lost large swathes of north+western India to invasions from the steppe.
You call GP's post "a lazy opinion backed by little research", but when you dig into the facts I can't see how you could argue that his claim is incorrect. The British Raj alone seems to qualify as the longest-lasting unification of India before the modern Indian state, and if you include any part of the EIC's rule then it's indisputably so.
Mughals did not unite the subcontinent at any time. Even at its peak, Kerala, parts of Tamil Nadu and Sri Lanka was out of its influence.
Parts of Kerala became British territories after 1804.
“In fact, Buddhism, which had flourished in Bharat for 1600 years, suddenly vanished almost completely as soon as Muslims became masters of Delhi and started raiding the plains of Ganga.” Citation needed?
1) https://en.wikipedia.org/wiki/Decline_of_Buddhism_in_the_Ind... - From 986 CE, the Turks started raiding northwest India from Afghanistan, plundering western India early in the eleventh century. Forced conversions to Islam were made, and Buddhist images smashed, due to the Islamic dislike of idolatry. Indeed in India, the Islamic term for an 'idol' became 'budd'. — Peter Harvey, An Introduction to Buddhism ... According to William Johnston, hundreds of Buddhist monasteries and shrines were destroyed, Buddhist texts were burnt by the armies, monks and nuns killed during the 12th and 13th centuries in the Gangetic plains region. The Islamic invasions plundered wealth and destroyed Buddhist images ... The decline of Buddhism in the Indian subcontinent coincides with the spread of Islam in that part of the world, especially due to the Islamic invasions that occurred in the late 12th century. See sections "Turkic Invasions" and "Decline under Islamic Rule".
Buddhism was the tranquilizing death of India. You can argue that Islamic invaders would have conquered India anyways - but with Buddhism they rarely even had to fight!
Their puzzlement is even captured in several journals where they could range for hundreds of miles and loot/burn with little to no resistance. And do it once again a few years later!
There is a stronger argument to be made that it was because of the establishment of Buddhism as the de-facto state philosophy/religion/practice in North/Northwest part of India that the Islamic invaders could conquer India. Buddhism for all its intellectual/ethical/moral strengths was not a pragmatic religion. It ignored the realities of Life in favour of higher ideals in a context ill-suited to its survival and hence paid the price at the hands of barbaric muslim invaders. This happened through the elevation of Ahimsa into an all-encompassing tenet of state policy which severely sapped the Martial Spirit of the population and thus could offer no resistance to invaders bent on genocide. Prior to Buddhism (and Jainism) while Ahimsa was considered one of the central pillars of Hinduism its limitations in the practical world were acknowledged and Kings were expected to protect by force if necessary, those practicing Ahimsa as a way of life. With this gone, North/Northwest India was easy prey to barbaric muslim invaders who did not play by the same rules.
During their conquest of Sindh, the Arabs brought the non-Muslims into the category of ahl al-kitab, considering them ahl al-dhimmah (protected subjects) and thus practicing a certain amount of non-interference in their religious lives under the condition that they fulfil a number of obligations that came with this status. Since both Buddhism and Hinduism are literate religions with scriptures, the precedent of assimilating Zoroastrians into the category of ahl al-kitab was extended to them as well. The dhimmis were obligated to pay the jizya for following their ancestral religion. The historian Al-Baladhuri notes a decision by Muhammad bin Qasim in relation to a Buddhist vihara and Aror that after conquering the city through a treaty (sulh) he agreed not to kill the people and enter their temple, in addition to imposing kharaj on them.[29] The Buddhists had petitioned the Arabs for the right to restore one of their temples and it was granted by Al-Hajjaj ibn Yusuf. However, this decision was later violated by the Pact of Umar and subsequent Muslim law codes which prohibited the restoration of existing non-Muslim religious structures as well as the building of new ones. Despite this fact, Buddhist inscriptions were still being recorded in the eleventh century.[28] Some Buddhists also fled and emigrated from Muslim-ruled areas into other regions. Unlike Brahmanical worship, Buddhism rapidly declined in Sindh after the eighth century and it virtually disappeared by the eleventh century.
You've broken the site guidelines repeatedly and badly in this thread. We have to ban accounts that do this, so if you'd please review https://news.ycombinator.com/newsguidelines.html and stick to the rules when posting here, we'd appreciate it.
That means no personal attacks and no religious flamewar, among other things.
I don't doubt that you know a lot about this and other topics but we need you to make your substantive points thoughtfully and respectfully.
I understand your point (the letter) but disagree with its spirit.
One should not tolerate attempts to intentionally "sweep under the rug" documented genocides and distort History just because it involves someone's Religion (their in-group). It is easy to be blind to genocides if one is not forced to face up to them, admit their faults and change their ways. Else the vicious cycle keeps spinning to the detriment of Society as a whole. Hence my forceful attempt to show up a person who intentionally was downplaying documented genocides. Note that most of my data/articles are from wikipedia (curated database and hence less susceptible to fake news/specific narratives/gaming) and not some opinion piece to push a narrative.
As you are very well aware, there are insidious groups trying to game the system at HN (and elsewhere) to push their narratives. They are not interested in the Truth/Factual Data/Social Accountability etc. but are only interested in distorting reality to their benefit (see Orwell's essay on Nationalism). These people/groups need to be called out forcefully even if it means not obeying all rules of etiquette. It is in that spirit that i wrote my comments.
> there are insidious groups trying to game the system at HN (and elsewhere) to push their narratives.
This sort of perception is common and has been common on HN for well over a decade, but I've rarely seen any evidence to support it. What there is evidence for—plenty of it—is users with different backgrounds misperceiving each other's comments as astroturfing/shilling/etc. because they simply can't imagine anyone holding those other views in good faith.
The odds are high that this is what you're encountering. It's not some shady misinformation group; it's simply people with very different backgrounds than your own, who hold opposite views for legit reasons, just like you hold your own views for legit reasons. These are difficult historical topics that there's no consensus on.
Here are a couple of long explanations I posted about this in the past:
The Minority Rule, often associated with Nassim Nicholas Taleb, refers to a principle in which a small, intransigent minority can have a disproportionate impact on the behavior of a larger group, eventually leading the majority to adopt the preferences or practices of that minority. This occurs because the minority is highly committed to a particular preference or practice and is unwilling to compromise, while the majority is more flexible and willing to accommodate the minority's demands to avoid conflict or inconvenience.
Key Points of the Minority Rule:
Intransigence: The minority is unwavering in its position and refuses to accept alternatives.
Flexibility of the Majority: The majority is more flexible and often prefers to avoid confrontation or inconvenience, leading them to adopt the minority's preference.
Asymmetric Impact: Even though the minority is smaller, its rigid stance can lead to a situation where the majority conforms to the minority's preferences.
Examples:
Cultural Practices: In a mixed group, if a small number of individuals strictly follow a particular dietary rule (e.g., kosher or halal), the larger group might choose to accommodate these restrictions, leading to everyone adopting the more restrictive practice.
Regulations and Standards: Sometimes, a regulation or standard that applies to a small subset of people (e.g., accessibility requirements) becomes the norm for everyone because it’s easier or more efficient to have a single standard.
Implications:
The minority rule highlights how committed minorities can exert significant influence over larger groups, often shaping social norms, practices, and even laws. This can be both positive (e.g., ensuring certain ethical standards) and negative (e.g., stifling diversity of thought or practice).
Nice writeups. While most of your reasoning/logic are valid i think you are missing a few crucial viewpoints which should be incorporated into your "HN filtration and decision-making" process.
I presume you know of Nassim Taleb's "The Minority Rule", if not see his article The Most Intolerant Wins: The Dictatorship of the Small Minority - https://medium.com/incerto/the-most-intolerant-wins-the-dict... and video explanation https://www.youtube.com/watch?v=MwlW2aamDFc Any system can be gamed by an intransigent group by applying this rule under the guise of victimhood/false equivalence/even-handedness/appeal to authority/religion/PC/DEI/etc. Various language techniques like phrasing/tone/insinuation/instigation/support/oppose/etc. can be used to lead/sway/hint/push towards the group's viewpoint irrespective of Truth/Reality. In today's world all Human topics involve Politics/Propaganda/Manipulation/Spin/Gaslighting/etc. whether we like it or not. The effects of "events" (eg. HN comments) in these domains are non-linear (pareto/power law/etc.) and hence a single outlier can ruin everything i.e. you don't need an actual "shady misinformation group".
I am not sure how HN does its moderation but i can guarantee that the above is happening in one form or another. I have seen this in threads to do with Russia-Ukraine war, Israel-Palestine issue, Boeing issues etc.
As an example, you say; "These are difficult historical topics that there's no consensus on." which is factually incorrect given the wikipedia links i had posted. You have been manipulated to disregard Truth in the guise of even-handedness :-)
You realize I belong to an Ahmedi family? What kind of insidious “ingroup” is that in Pak context? Please tell that to any Pakistani who will collapse in peals of laughter.
If you want to talk down to someone who was born and brought up as one that’s your prerogative but you’re the one who’s looking stupid. Yes, your spelling is the “official” one.
You can think what you want. I tend to worry about people telling me my parents should be assassinated rather than which vowel to use (this spelling issue obviously doesn’t arise in Urdu)
As an Indian, it is extremely naive and childish to dismiss any consideration of the British rule as a "slap in the face". The British introduced electricity, railways, capitalism and a thousand other things we take for granted behind those saffron-tinted glasses.
Hell, the British Raj was what unified India into a single national identity. It was more fractured than the European continent otherwise.
- Most of the abstracts are written by the author of the paper, so might not be as unbiased as an actual "community-written" abstract.
- There's no stated guidelines for the "community-written" abstract e.g. should it be less biased than the original abstract, should be shorter than the original, should it be more accessible to a less AI crowd or all of the above.
- There's no way to upvote/downvote some abstracts e.g. the "attention is all you need" paper has two abstracts and one of them is clearly worse than the other.
Hello! You are on point, building community to write the abstracts, while also setting up relevant guidelines are my main focus for now.
I will think of how to communicate guidelines and expectations to the content in a clear way. Thank you!
Upvote/downvote is available for logged users. As you pointed out, it's not visible when you're not logged in yet, so it would make sense to show score and buttons to anonymous users as well
AI rescued Nvidia when nobody was buying their shovels for digging crypto gold. If it wasn’t for ChatGPT, Nvidia story would have been very different right now. He is probably just hoping that this dream never ends. Otherwise there’s no way to justify the current valuation in the long term.
Another problem with the title: the article is about DPO, which doesn’t do reinforcement learning. So not RLHF. I guess RLHF has more of a name recognition than DPO.
This was discussed in another comment, DPO is pretty much strictly better than RLHF + PPO, and far more stable when training. Yes, DPO is not technically "RL", but it's semantics for the most part. DataDreamer does support PPO training if you want, but it's so unstable, it's a less popular choice now.
In the DPO paper linked from the OP page, DPO is described as "a simple RL-free algorithm for training language models from preferences." So as you say, "not technically RL."
Given that, shouldn't the first sentence on the linked page end with "...in a process known as DPO (...)" ? Ditto for the title.
It sounds like you're saying that the terms RL and RLHF should subsume DPO because they both solve the same problem, with similar results. But they're different techniques, and there are established terms for both of them.
I think the discussion in the other comment thread discusses this well. They are different techniques, but the line between RL & SL is quite fuzzy. The DPO authors advertise this as a "non-RL" technique to precisely get away from the reputation of unstable training RL has, but they also say and treat the language model as an
(implicit) reward model, similar to PPO. The point is well taken though, I will update this page to clarify the differences to avoid confusion.
> DPO is pretty much strictly better than RLHF + PPO
Out of genuine curiosity, do you have any pointers/evidence to support this. I know that some of the industry leading research labs haven't switched over to DPO yet, in spite of the fact that DPO is significantly faster than RLHF. It might just be organizational inertia, but I do not know. I would be very happy if simpler alternatives like DPO were as good as RLHF or better, but I haven't seen that proof yet.
Because a salesman’s skills complements those of a researcher. Salesman sells what the researcher built and brings in money to keep the lights on. Researcher gets to do what they love without having to worry about the real world. That’s a much sweeter deal than a micromanaging PI.
I used to work on a production auto-complete system operating at over 100k peak QPS. For prefixes of length one and two we would not even bother hitting the server, just from a quality perspective, not because of latency/throughput considerations. Btw, up until 3 characters, you could store everything in an in-memory hash map. 20x speedup on length 4 and 5 prefixes is still very impressive, but not quite 1000x speedup either.
I also worked on a production auto complete feature for a web app a bit ago and I couldn't agree more with the quality sentiment. One or two characters is almost never enough to give a meaningful result. Using history or similar user search is much more effective than trying to guess what someone meant by "th".
I have been training a natural intelligence model for 3 years now and she still doesn’t get nuance. Things are either good or bad in her book: nothing in between. My plan is to let her train with binary good/bad labels till the age of 5 and then start smoothing the labels after that. Wonder if that works for your AI.
Related trick: I found that training two Natural Intelligence (NI) models in parallel, and having them train each other for most of the time, leads to significant leaps in capabilities. Notably, when one NI picks up a skill, it often results in spontaneous transfer learning - the other NI picks that skill up very quickly, much faster than it would through direct training.
This scales well, too. There are facilities that provide services of co-hosting and cross-training up to ~two dozen NI models in a shared environment - in my experience, this provides similar training benefits to running multiple NIs on your own, at fraction of the cost.
(The facilities are exploiting some neat economies of scale. Talking to some employees, I learned that the transfer learning and co-activation are embarrassingly scalable: if you get two-three NIs to pick up a thing, all the rest immediately follow.)
This took a couple reads, but it’s funny. The bad news is that I’ve been training mine for 17 years and nuance is still something that needs more training.
in my mind I've built an 'emotional engine' to add nuance to models understanding, take something like Plutchik's wheel of emotions and create a high quality multi-modal dataset based on that structure, given our current technology takes inspiration from the brain, it would seem like having discrete models specialising in particular aspects of 'intelligence' that are then organised into a mixture of experts is an interesting area to explore, and perhaps more accessible as smaller models require less resources.
I have code stubbed out for this in mitta.us. It has 9 states, based on the Plutchik wheel, with emojis for the states. States drive temp and a few other things and drop the state into prompts.
The accounts aren't wired up by default to the AI and I am refactoring the templating system right now, but you can definitely start storing and searching things.
