For what it's worth, I have used Vim since ~2004. When I built Light Table, one of the first things I did was put a Vim mode into it. I understand pretty well why at least I use Vim. I also believe that these days there's relatively little payoff given the cost of investment. Rarely is the speed of text-editing the bottleneck in building something real. Moreover, there are far more powerful ways to interact with code than at the text buffer level.
> Moreover, there are far more powerful ways to interact with code than at the text buffer level.
This is actually one of my core reasons for using Vim. I agree, but all of those "more powerful ways to interact with code" happen in my head, not in an IDE. And my head is far more flexible than the IDE. In pretty much every IDE I've used you start thinking according to the patterns of transformation that the IDE supports, which can be helpful for a beginner, but is ultimately very limiting. New languages start with no IDE support. This is mostly for practical reasons I concede, but it is usually also often the case that the new language is being written because it represents a new paradigm of thinking. It's not until the language matures and the common patterns of thought become known that they then become encoded as "helpers" in IDEs.
The other main objection to this, I think, is to point out that linear textual encoding of code is itself a bottleneck, and that a strict text editor like Vim will never let you move past that. Implicit in this assumption, however, is the belief that there is a better way to represent code than linear text. However, I would point out that throughout all of mankind's history, language (and text as it's persistent form) has been the highest form of communicated logical representation. We have had access to pictography and other forms of communication for millennia, but none has had the same ability to encode and transfer knowledge and logic like text. I think we are primarily adapted as humans to think in terms of language. My bet is that there isn't a more powerful abstraction than this given the structure of our brains. I'm happy to be proved wrong though.
Given both of the above, personally I find the most productive means of working is to work primarily at the conceptual/abstract level in my head, model the problem and solution in thought, then translate the solution into the most natural form for expressing thought: language (as text).
So most of the tools that the IDE gives me (especially regarding code transformation, etc.) are hard-coded, less powerful versions of the kind of mental processes I am already doing. I don't need those. What I do need is a way to encode my thoughts in text as quickly and efficiently as possible. I don't want to have to think about that at all. I want it to flow naturally from my mental model into prose (though not English prose, obviously). For that an IDE again gets in the way. It's slow and full of distractions. The Vim keybindings have become so ingrained that it's muscle-memory. And I've yet to find another paradigm that lets me manipulate text as effortlessly.
I think you are right, but: how about tools like Excel, which is very non-linear, and yet it seems to be easier to pick up, and less intimidating, for complete newcomers compared to traditional programming?
Excel is, to me, an example of an alternative means of arranging logic that is very well suited to a specific subset of problems. You'll notice that people aren't typically building general-purpose applications with Excel. It's clearly not a replacement for a general-purpose language. But it is also clearly a more "natural" fit for it's niche of problems.
Actually, I think that this line of thinking holds a lot of promise: building specific tools well-tailored to solving specific types of problems. I think we could easily find models and alternate modes of thinking that outperform text for given domains.
I guess my argument is that as humans language is our most fluent medium for communicating abstractions. In the general case I don't think it will be outperformed.
Sure, it might not be general purpose, but the "niche" is really quite large: there aren't too many websites written in Excel, but there aren't many written in C++ either :) I am really not a fan of Excel overuse, but it happens, so I think there must be something about it that makes its programming model attractive to people.
For one, constructs like loops that we have mostly internalized as natural, can look a bit intimidating to non-programmers (they are not found in regular language either though!). Even among fairly experienced coders, for example in statistical programming, people who come from general-purpose coding background sometimes find loops easier to read and write compared to "vectorized" code, and vice versa.
Thinking about it, the way say much Java code is written is not what I would call a linear language either, although that I suspect might be more a result of path dependence and optimizing for "large teams of replaceable programmers".
Bret Victor's group is still going (they got their own funding); I believe John Maloney (of Morphic/Etoys fame) is also still going with his own funding. The rest of us are all gone...
Perhaps, but it's pretty easy to make judgements from the sideline about what people should or shouldn't do.
I was the PM on Visual Studio. I can assure the things I showed in Light Table weren't there. Nor were they in Eclipse, as I studied that as well. You can pick and choose any number of things from our work over the years and say "Hey, but this looks like that" - I'm sure it does, but the question is does it work like that? Does it enable you the way the things we showed did?
> Light Table was also ultimately abandoned before completion.
Unfortunately even after paying ourselves only just enough to live in the area (around 40% of what my peers were paid), we needed to find a way to eat. No software is ever done, but we had more than 40,000 people using it and Light Table was a stated influence on tools at Apple, Google, and Microsoft. We did our best. Was there more to do? Of course, there always is, but at some point we had to make the hard decision to leave LT in the hands of the community. I'm curious what direction you think we should've gone instead?
> Perhaps Granger should rein in his ambitions somewhat
Maybe, but at the same time, we had people willing to let us try. You paid literally nothing for access to our work, nor to the effects our research had on others, so I'm not sure why there's this much negativity here. We need people testing the fringes because where we are is so far from where we could be. Lots of things did work in Eve and there were cases where we were so much more efficient it hardly felt like we were "programming" at all anymore. We've shared everything we've done and we'll continue telling more people about it in the hope that others can benefit from what we've learned.
And yet you're the one acting hurt, while I'm the one having to shutdown the project. That's very different from the HN that rallied us to do our kickstarter in the first place - the one that encouraged innovation and trying to do crazy things in the off chance that they work. There's so much more to do and I sincerely hope HN doesn't become so cynical and demeaning that it's not worth sharing people's efforts here.
For whatever it's worth, I backed light table on kickstarter and as far as I'm concerned it was well worth the money. I got an interesting prototype editor which has had a big impact on the wider ecosystem (the swift playground comes particularly to mind). I've gotten to read your consistently interesting blog and watch you experiment with different pieces of the language design space.
I can't imagine how hard it must be to shut it down after all these years, but I know whatever comes next in PL and IDE design will owe your project a huge debt. Thank you for everything!
> Graduate school is a great place for "people testing the fringes."
Testing, yes, but not exploring. Grad school gives you one experiment and then you have to start publishing. So if you have something you want to try out, with a direct implementation strategy and a clear set of possible outcomes, then you can do that at grad school.
If you want to try many things and iterate, grad school will not work well. You will be expected to publish digestible “learnings”, and so you will end up skewing your work towards ideas that are likely to produce compelling presentations.
There’s no good place to experiment, in either the corporate or academic world. Your best bet is to move between many domains, trying small ideas in context while also delivering value, and only ever doing your real Hail Mary experiments at home on your own dime.
If you don’t care if you have anyone in your lab, and you don’t have any expenses, you’re pretty free as a tenured faculty.
But if you want students,
you need to find grants for them. And if you need materials/travel/services/etc for your research that’ll require grants too.
I'm happy for all the work you've put out, and I agree we need to try and experiment more and innovate, and so I found Eve was a more interesting experiment. That said LightTable had more potential for success and impact. I still dream for a more polished and full featured LightTable editor. A Jupyter notebook on steroid usable for generalized programming. ProtoREPL in Atom has picked up some of it, but overall there's just not the man resources behind any of these to really flesh them out and that's sad.
How do you deal with consistency in the offline case? E.g. If I make a bunch of edits while disconnected, and others have made edits while connected, how are the conflicts resolved?
This works similar to the Realtime Database in that it's last-write wins (where in the offline case, "last" is the last person to come back online and send their write to the backend). This model is very easy for developers to understand and directly solves many use cases, especially since we allow very granular writes which reduces the risk of conflicts. But for more complex use cases, you can get clever and implement things like OT conflict resolution as a layer on top of last-write wins, e.g. similar to how we implemented collaborative editing with www.firepad.io on the realtime database.
Providing a one-size-fits-all solution here is probably impossible, but it seems like it would be nice to provide some mechanism to be notified that you're making edits based on stale information. If such a mechanism existed, it would be easy to add a bunch of canned merge strategies. In doing so you can probably teach people a little bit about the pitfalls they're likely to run into (these sorts of bugs are insanely difficult to track down), while not really making them do much work.
The approach we've taken in Eve is that we can't solve all these problems for you, but we can at least let you know that things can go sideways and prompt you to make a deliberate decision about what should happen. It's amazing how helpful that ends up being.
Thanks for the feedback. I think you're right and we're interested in exploring what we can do to help people more in the future. One of the really nice things about Cloud Firestore is that documents are versioned with timestamps in such a way that we could definitely detect and expose conflicts and let you decide how to deal with them... It's mostly a matter of identifying the common use cases and then figuring out the right API to make them possible without going too far into the deep end of conflict resolution.
It looks to me like Firestore's API doesn't include a "default" way to upload user edits to documents. Conflict detection is possible using the transactions - https://cloud.google.com/firestore/docs/manage-data/update-d... - you can do something like HTTP's PUT If-Unmodified-Since (or PUT If-Match).
Good point. read-modify-write transactions are a good way to detect conflicts and get a chance to handle them, but they're unfortunately limited to while the client is online. If the client is offline, the transaction will fail so they're not useful for general conflict resolution. This was an intentional decision because there's not a straightforward way to preserve the intent of the transaction across app restarts. But there may be options for adding some sort of conflict resolution strategy in the future that leverages the same underlying primitives that transactions use today.
Is there anything written about your multi-way join algorithm? We've done some really interesting implementations of Generic Join and the like and are always interested in hearing about other potentially better join algorithms.
It's only described in my thesis, which is not yet publicly available. (It should be, but the university thesis publication process seems to have a latency measured in years...)
The case for triangle joins is simple to describe, though. Suppose we have attributes A, B and C and finite maps (relations) R : A x B -> X, S : A x C -> Y and T : B x C -> Z. Now build tries for each so we have
abx : Trie A (Trie B X)
acy : Trie A (Trie C Y)
bcz : Trie B (Trie C Z)
We can join two tries using join : Trie k v1 -> Trie k v2 -> (v1 -> v2 -> v) -> Trie k v. Putting this all together we have:
Compared to a database that doesn't know how to handle triangle joins the difference is absolutely stunning. The above one-liner managed to do in 5 seconds what took MySQL over an hour...
Is there any way I could get a preprint of your thesis?
I've been working on a project that sits somewhere between spark and riak and this actually might address a core issue I have both at the node and query dispatch level.
Hmm. Just checked again. Two of the expressions match neither word, one matches both, and one matches only one.
EDIT: I see. The confusion is over partial match vs whole match. I thought whole matches were standard when talking about regular expression in the abstract, but looks like I was wrong. Just changed the problem to include the anchors. Thanks for pointing this out.
That's not the standard meaning in my experience. Regex's are often used for extracting information from larger strings, forcing a full match by default would be pretty counter to their purpose.
That would also make this question specific to python, which wasn't at all what I was thinking about. It's a question about regular expressions, it should be as agnostic as possible to implementation.
Yeah, there are a lot of interesting benefits to synchronous programming that haven't been explored in a wider context and we're excited to be able to do so. Figuring out how to actually implement Eve's semantics has been quite a quest and unfortunately the implementations of most of those languages don't really fit us. Fortunately, we've put some really interesting things together lately that have produced some very surprising performance numbers for us, so hopefully that's finally resolved and we can move on to how GALS and the like apply in our world. :)
That must be a different Aurora. The one mentioned here was never really released, only demo'd at Strange Loop. We subsequently went on to turn that into Eve[1].
Thanks for confirming that. This looks very interesting. How active is development of the mathematical side of things? Also have graphics capabilities been developed yet or are they far away? The HN article indicated Aurora now Eve would have these capabilities. I'm interested because of the similarities I see between Eve and other literate programming approaches such as the Jupyter project, Mathematica, Sage notebooks, R motebooks. Eve looks like it will soon be joining that group.
Recent development has been focused on shoring up the semantics of the language, making the runtime more performant, and packaging our syntax and editor to make it more usable for a wider audience (i.e. people outside our office). Efforts relating to a UI that is approachable from a non-programmer perspective have been waylaid until we produce something that is usable and comfortable for at least the programmer crowd.
Is there any chance that you'd ever open-source the original Aurora demo? Seems like it would be a great foundation for further exploration in this space.
Eve[1] is in the process of becoming a lot more real than it has been up to this point. We've found a great model and discovered a way to build a high performance implementation of it, which means much of the foundational research is finally in place. Over the next couple of weeks, we'll be revamping our website, docs, etc to help people get started building real things with it. :)
There's going to be a lot of really exciting stuff coming over the next few months. We've gathered a set of ideas, evidence, and implementations that have certainly blown us away - we hope others will find it valuable too.
EDIT: I realized I didn't address your initial question. Fwiw, we just recently crossed a really big milestone in terms of usage - more than 40,000 people have now played around with Eve on http://play.witheve.com and we've learned a ton from that experience. Part of the website revamp will be making that workflow simpler and nicer. We have a surprisingly high conversion rate (> 30%), so hopefully we can help smooth out that experience and begin to grow the community more and more.
I can't load the site on my phone. Mind pointing out what makes Eve so special? I'm suspicious of us figuring out anything mind boggling new at this point and assumed Eve was all a gimmick (I hope to be proved wrong :)). Something like Red makes a lot of sense to me as that ahhh language as it is tiny with no install, can be used for high/low level coding, has excellent DSLs such as GUI builders, and can compile to a native binary to name a few things that kind of shocked me. Eve seems to be more like an online Smalltalk? It's always fascinating to hear new ideas and I wish the best for your project!
We've certainly taken a lot of inspiration from Smalltalk, but I think the semantics we've arrived at make a really nice programming environment, with some surprising properties you may not think are possible.
Eve has a similar philosophy to Red/Rebol - that programming is needlessly complex, and by fixing the model we can make the whole ordeal a lot nicer. We start with a uniform data model - everything in Eve is represented by records (key value pairs attached to a unique ID). This keeps the language small, both implementation-wise and in terms of the number of operators you need to learn.
Programs in Eve are made up of small blocks of code that compose automatically. In each block you query records and then specify transforms on those records. Blocks are short and declarative, and are reactive to changes in data so you don't worry about callbacks, caching, routing, etc.
Due to this design, we've reduced the error footprint of the language -- there are really only 3 kinds of errors you'll ever encounter, and those mostly relate to data missing or being in the wrong shape that you expect. What's more, we'll actually be able to catch most errors with the right tooling. You'll never experience your app crashing or errors related to undefined/nil values.
We've made sure your program is transparent and inspectable. If you want to monitor a value in the system, you can just write a query that displays it, as the program is running. I like to think of this as a "multimeter for code". You can do this for variables, memory, HTTP requests, the code itself ... since everything is represented as records, everything is inspectable.
Because at its core Eve is a database, we also store all the information necessary to track the provenance of values. This is something most languages can't do, because the information just isn't there. So for instance if a value is wrong, you can find out exactly how it was generated.
There's a lot more work to do, but we have big plans going forward. We plan to make the runtime distributed and topology agnostic, so you can potentially write a program that scales to millions of users without having to worry about provisioning servers or worrying about how your code scales.
We're also planning on multiple interfaces to the runtime. Right now we have a text syntax, but there's no reason we couldn't plug in a UI specific editor that integrates with text code. We've explored something like this already.
Anyway, those are future directions, but what we have right now is a set of semantics that allow for the expression of programs without a lot of the boilerplate and hassle you have to go through in traditional languages, and which provide some unique properties you won't find in any other language (at least none that I've used).
Thank you for these intriguing thoughts behind the development of Eve. This project was for me the most valuable find in this whole discussion. There are a number of fundamental design decisions in Eve that opened my eyes to a fresh rethinking of the underlying assumptions in existing languages.
The code examples demonstrate surprising simplicity in achieving features that would be complicated to implement in other languages. I'm convinced that Eve will influence how programmers think (at least it did for me) and promote development of languages/frameworks/libraries that adopt some of these ideas. Great work, will be following with interest.
Thanks for the long description! It is most appropriated and helped me get a feel. I agree wholeheartedly that programming needs to change. Best of luck! I'll be keeping an eye out on Eve.
