Hi HN, we're the team behind Ente Photos, and we just published a deep-dive into how we built private, on-device ML features for photo discovery - an effort that's been in development for over a year.
What makes our approach unique is that we're running sophisticated ML features (semantic search, face recognition) entirely on users' devices while being fully open-source[1]. All ML results are then synced across devices using end-to-end encryption[2] - an approach we haven't seen anywhere else in the industry.
Building this was complex given the different platforms and devices we support, combined with limited compute resources. We had to write numerous custom implementations for image processing and rebuild data science tools from scratch to work across different platforms.
Our users can search their photos using natural language or find pictures of specific people, all while keeping their photos private[3]. No data ever leaves their device unencrypted, and the ML processing happens entirely locally.
What makes our approach unique is that we're running sophisticated ML features (semantic search, face recognition) entirely on users' devices while being fully open-source[1]. All ML results are then synced across devices using end-to-end encryption[2] - an approach we haven't seen anywhere else in the industry.
Building this was complex given the different platforms and devices we support, combined with limited compute resources. We had to write numerous custom implementations for image processing and rebuild data science tools from scratch to work across different platforms.
Our users can search their photos using natural language or find pictures of specific people, all while keeping their photos private[3]. No data ever leaves their device unencrypted, and the ML processing happens entirely locally.
Happy to answer any questions!
[1] https://github.com/ente-io/ente
[2] https://ente.io/architecture/
[3] https://ente.io/blog/machine-learning/