The reason why I've rooted for Apple all these years is the shared values.

Sure, they make great products and they are immensely successful at it, but its their ability to go against the grain, to stand up for what they think is right when everyone else is doing something else, is something I aspire in my own life and work.  Their success feels like an affirmation of my values more than any thing else.

Be it their stance on data privacy, or their recent outlook on AI with their focus on building, and promoting tools that empower human expression rather than replace it, or their focus on Accessibility; more often than not I find myself agreeing with them in principle rather than looking at them as yet another soulless capitalistic enterprise where immediate profits reign supreme.

And this is consistent across every aspect of the company. What's amazed me over the years is how egalitarian they are.

I obviously owe a lot of my career success to Apple, and Apple's products, one thing that's always struck me is how there's literally no difference in how my apps as an indie developer and those of industry stalwarts have been treated.

Apple's prestigious annual design awards end up awarding many indie developers each year too.

As an app maker, being put on the same pedestal as industry giants from day 1 is immensely empowering (and why would it be any other way!?).

Through their actions, Apple makes it clear that they see no difference between indie developers and larger enterprises. Unlike other places where the systems and incentives in place naturally award those with bigger pockets.

Over the years, I've enjoyed using Apple products, agreed and disagreed with their design decisions, held strong opinions on where I'd change things, but ultimately it's shared values that makes me come back to rooting for their successs.

Here’s an experiment. Take 5 unique shapes. They look cool right? Each one stands out on its own has unique characteristics, texture, and color.

We should have more of them. So let's multiply them. Maybe we can add new colours too.

At an even bigger size, the textures are not even clearly noticeable, so let’s remove them.

Let’s make it a little bigger. Actually the colors are making it look incongruous, so let’s have the same color.

Let’s make it even bigger. It’s not very efficient use of space. We can pack way more if everything is a circle.

Great! Now we can scale this to a million units! A million circles of same size, spaced in a nice grid it is.

This is exactly what every aspect of technology looks like right now. You want a phone, it’s a rectangular slab of aluminum. Colors? Barely distinguishable variants of gray, and black. You unlock the phone, every icon is just text slapped on to a white background or an abstract shape that is wearing the brand identity uniform.

You open the phone, every app is a collection of white rounded rectangles and black text.

In fact, we’re in such stages of scaled homogeneity that even language is become a sum of averages thanks to ChatGPT (an LLM is language at scale!)

Scale is the enemy of creativity. Indie apps that have to make an app for one platform, and do it well, have much more fun app icons and interfaces than bigger players. Pre-industrial revolution furniture looked nothing like the sleek boring industrialized furniture of today because billions of units weren't being shipped to people.

When conformity is the norm there’s little room for soul, creativity or fun. Fun is aberrant, uncouth, rebellious. This is what using technology feels like today. It’s scaled to a billion people, but those billions of people are being drowned in notions of conformity, some dictated by mass production, others by product managers and dysfunctional software economies.

No wonder it’s boring and lacks soul. When you’re making something for everyone, you’re really not making anything for anyone.

The modern tech discourse is muddled with buzzword technologies being referred to as emerging technologies, collectively. So much so that when we think of emerging technologies, we fail to look beyond things like AR Glasses, Mixed Reality Headsets, or more recently "AI hardware" or "Agentic AI".

But that's not the limit of Emerging Tech Design. In fact, Designing yet another "AI hardware" or somewhat better "AR Glasses" completely misses the point of designing emerging technologies.

The Form Fallacy

Here's the harsh reality, a new form-factor isn't going to make AI hardware suddenly more useful. Emerging technology design isn't just about giving technology a new kind of input and output interaction.

Mixed Reality Headsets, Projection based interfaces, AR Glasses, AI Chatbots, are novel methods of interaction, that combine existing technologies into a sensor—actuator system (or input or output in the case of software) to create a new experience. They are a new form of housing for technology. Making them doesn't automatically make them desirable to people.

The Vision Pro presents a vision of spatial computing but fails to provide people a reason to buy it over a traditional computer. It doesn't really solve a pressing need in people's lives.

The Ai Pin failed, not because the form factor wasn't ideal, it failed because the device makers had no idea who their user was and so the product was little more than a string of cool demos at its best.

Emerging Technology is technology being used in a novel way to solve human problems. so that they spark our imagination of a different, better future. What gives them the "emerging" status is the promise of significant changes in people's lives through a new medium.

When the focus is on people and their needs, it seems counter intuitive to begin by wrapping a new technology in hardware. To reach a form, one must go through the process of designing it.

To design something you must know who you are designing for, and what is the problem to solve, before you can really finalize a form.

The Design Process

We start with the people, their wants and needs (the problems they are facing) and the actions they perform to achieve them.

The Form Smart Swim Goggles are the perfect example of how emerging technologies like AR prove to be useful when starting with a very clear user (swimmers in this case) and addressing their problems.

How We Arrived at Smart Swim 2 - What Went Into Developing Our Next-Gen Goggles

During my 4.5 years leading the product function here at FORM, it has been incredibly exciting and inspiring to watch the purpose of the company evolve to where we are today, knowing that we exist to elevate athletic performance through real-time visual feedback. Athletes of all types and abilities…

FORMPaul Hossack

Well, That's simple enough. So why don't we have more products that do this?

The reality is that each intervention results in intended and unintended consequences rather than “solving the problem”.

But before we focus on our intervention creating a feedback loop, let’s take a step back and look at the problem itself, and why trying to solve it is so complicated.

The problem

Our environment, circumstances and experiences shape our wants and needs, which are fulfilled by the appliances around us.

But these wants and needs, or the appliances that are designed to fulfill them don’t live in isolation. They’re part of bigger problems, that are being continually being shaped by our actions, appliances and the world around us.

Each Want and need points to a problem, that problem no matter how straightforward it may seem, is a part of a bigger more wicked problem. How do we tackle them then? It seems as if no new product (or emerging technology) is good enough at solving these seemingly untamable problems?

A quick primer on Tame and Wicked Problems.

There are no Tame Problems, Every tame problem lives inside a bigger wicked problem. To tackle a wicked problem, we slice it into smaller, somewhat tamable problems.

A Thin Slice of a bigger problem

Now that we’ve scoped out our problem. Let’s descope again. This time, by creating thin slices of the wicked problem. Identifying people, their immediate issues and how they connect to the bigger problems.

Once we have scoped out our problem as a complex system of elements and interconnected relationships, we find boundary objects to define the bounds of the thin slice of the problem we want to tackle.

With this thin slice defined. We must start our usual design loop. Person, Action, Goal, defines the solution we’re working.

Once we have a fair idea of the people, their actions and their goals. Each task can be divided into input information, the logic, and output logic. This forms the “How” of how a problem is solved.

Once clearly defined we have a good blue print for what we need to design our emerging technology interventions for. The goal of the intervention should be to better help people achieve their goals. Just like a scissor helps people cut paper, or eyeglasses help people see.

The Interventions

But how do we design the tools that perform the action? How do we design the “emerging technology”. There are many different ways of going about it. I would like to restrict today’s discussion to three methods.

1. Creating what's possible and practical for today
2. Vision Driven Design
3. Creating a speculative future

Creating what's most possible and practical for today

Time Frame: 1-5 years
The best approach for designing products that will launch on a 1-5 year time scale. They use existing systems to build a new solution.

To design products in this space the most practical way to start is to use existing frameworks like the DVF Framework, or the DICE Framework, or MIT Three Lenses of Innovation, amongst many others in order to deeply understand the problem space, and develop a design, tech and marketing strategy.

Remember here you have to ship something quickly, so your prototype solutions must know exactly how something is going to come to life, and how it's going to make money.

The DVF Framework is one of the many frameworks that can be used to identify the  who is going to need this product (desirability), how it's going to be built (feasbility), and the financial viability of the product (viability).

The DICE Framework and other similar frameworks are great for applications on pre-defined platforms like VR. They answer the question of "What kinds of applications would be cool for this platform?". In case of the DICE framework, it lays out clearly the kinds of experiences that would be great for VR headsets.

MIT's Three Lenses of Innovation frame the design problem as something that sits in the political, cultural and strategic contexts. It provides an alternative approach to DVF and DICE frameworks.

The Gatorade Sweatpatch is a great example of how to create novel solutions using emerging technologies (micro-fluidics in this case) using existing systems (the patch doesn't require anything but the smartphone!)

Gatorade Gx Sweat Patch for Athletes | Smart Design

Gatorade’s Gx Sweat Patch and companion app are designed to bring sweat testing and personalized sports fueling recommendations to all athletes.

Smart DesignChris Hintermeister

Vision Driven Design

Here designers define a long term vision for how they want to see design in a certain problem space evolve, take a principled stance, build out prototype of their vision, followed by a highly iterative process to reach to a good design.

This is a "making-intensive" and "learning-intensive" process of creating a prototype of the future and tweaking it based on user feedback. It’s driven by a personal vision of how technology should progress (e.g., screen-less computing).

Examples of Vision Driven Design Practices

Bret Victor's Inventing on Principle is a similar design practice. Where Victor encourages designers to form a core principle for their work. One that's well articulated with a clear demarcation between right and wrong.

To explain his concept, Bret gives several examples of how inventors have followed a guiding principle to design their ideal world, leading to some profound inventions.

Douglas Engelbart's principle was to augment human intellect, where "knowledge workers" came together to build on each other's collective intelligence. Engelbart invented much of modern computing through this one vision. From Natural Language Search, to the Computer Mouse, to Collaborative Computing.

Alan Kay viewed computers as a medium to "bring new ways of thinking through computers". His work led to the creation of the world's first object oriented programming language, the Dynabook (spiritual predecessor to the iPad), and much more user research at Xerox Parc.

Bret's own principle is that "Creators need an immediate connection to what they're creating" . His work led him to make Dynamicland, a community computer space that enables people to bring and interact with real world objects in a new computing media.

Most Vision Driven Designs fail because there’s obviously a gap between how we imagine things should be and how the things actually end up playing out. That’s inevitable. What's not inevitable is designers through iterative design, designers can tweak their vision and focus to align with what people truly appreciate.

With Vision Based Prototyping, Iteration is the most important step.

It’s in such a space that we get to explore new technologies in unique ways. I personally enjoy this space, as it helps me bring out my ideas in the world.

The Thought Lamp is a smart lamp, that acts as a student's ambient study companion. It's a computer that's only present when you need it, and in the background when you don't. It is based on my vision that technology must augment reality, not emulate it, and so it only presents itself when it can augment my reality.

Share Wagon is a community food sharing system, designed to tackle the problem of food waste in apartment complexes. Designed by me and my friend (Aylish Turner), it was based on our collective vision of community building through sharing.

Speculative Design

Time Frame: 50+ years
Speculative Designs are provocations that further the design discourse. This process leads to artifacts that are essential to jog people’s imagination on where the use / misuse of a certain technology might lead us in the distant future. They highlight an urgent need for change, rather than being the change itself.

Very useful if you want to raise awareness for wicked problems not clearly understood by society today.

The Future Cone serves as a useful way of looking possible, plausible, probable and preferable futures.

Uninvited Guests (by Superflux) critiques the ubiquity of "smart" appliances, through a day in the life of a 70 year old sometime in the future. The short film through its speculative design and artifacts explores themes of isolation, overuse of technology and a society obsessed with homogeneity and perfection.

Uninvited Guests — Superflux

Uninvited Guests is a short film that explores the frictions between an elderly man and his smart home.

SuperfluxBy

The Faraday Chair (1995, by Dunne and Raby) is a speculative artifact designed as a retreat from all the electromagnetic emissions generated by technology when it eventually becomes ubiquitous. The chair is supposed to give people "psychological comfort by providing sanctuary".

While the idea of a tech detox may not appear to be completely speculative today, 30 years ago, this was unthinkable, and the strong, provocative visuals sparked curiousity and interest in the matter.

Faraday chair | V&A Explore The Collections

Faraday chair, designed by Dunne & Raby, 1995; comprising of an oblong transparent orange acrylic box close fit to a clear acrylic base suspended in a steel frame with legs at the corners. Piercing at the top of the box near one end is a free-hanging silicon tube attached to a mouthpiece at the low…

Victoria and Albert Museum: Explore the CollectionsVictoria and Albert Museum

We have an artifact that solves a clear human need. What next?

Appliances (or tools or design interventions) are great ways to tackle small challenges. As we've seen through our process.  A toaster is great solving for the need to have fresh toast at breakfast, but not much beyond that.

Remember, we started with a bigger problem, and de-scoped it into a very thin slice? So we've really just solved for a very small and specific need. The bigger problem is still largely unsolved.

We’ve solved a thin slice, what about the rest of the slices? We can take two different approaches here.

First, Product Growth (Vertical Integration) we increase our slice of the problem space and continue to grow infinitely. We grow our tool to encompass more problems and gradually get a bigger slice of the pie. Most traditional products have worked this way.

The problem with this approach is, unintended consequences add up as the product grows and result in adjacent wicked problems. For example the proliferation of the smartphone as one device to for all our computing needs has resulted in the adjacent issue of it amplifying loneliness and isolation through addictive apps.

Platforms, Protocols and Systems (Horizontal Scaling)

The second approach is, we acknowledge there’s only so much a product can grow until it becomes bloated and dishonest to its original purpose.

And we create a system that allows other slices to interact with our solution. Creating a fully formed puzzle where everyone comes together to solve the problem. The solution space still continues to grow, except this time, one company doesn’t have to do it all.

We create a platform and frameworks that enable more people to hook into our intervention and build out. Thereby delegating the process of tackling a wicked people to like minded people.

Tools that allow hooks into other solutions become platforms rather than appliances.

And before you know, your emerging technology product is not an appliance, it’s a platform that talks to and helps other talk to it to solve bigger problems.

Examples of systems

Remember the Gatorade Sweat Patch? It's not just a one-off product. It's part of the bigger Gatorade System that enables athletes to monitor their fitness.

The Gatorade Gx System

The Gatorade Gx is a good example of a vertically integrated system that aims to solve for an athlete's fitness monitoring using a range of low and high tech solutions.

The Lego Smart Play System

The newly introduced Lego Smart Play bricks are a set of primitive tangible technologies like a positioning system, a smart brick that coordinates with minifigures and other nearby objects to enable new experiences.

This is a great example of using emerging technologies. Lego built out a smart brick system to allow kids to have interactive gameplay sessions without looking at their iPad screens, and they didn't just stop at a standalone system, they literally built the building blocks that would enable kids to show their creativity.

IDEO's Food Waste Alliance

When IDEO started tackling food waste a little over 10 years ago, their big epiphany was that food wastage is a far bigger problem than what a single product could solve. So they started building out a system to tackle food waste which led to the Food Waste Alliance.

What the sustainable food future we’re working toward looks like

Over the last decade, we have been working with people, companies, governments, and organizations across the food industry.

The iPhone is a case study in emerging tech design

The iPhone is a perfect example of how a product started with immense clarity in who it were serving and what it was solving for, and then quickly expanded into a platform that anyone could build for. Unlocking new possibilities and capabilities that the iPhone excelled in.

The iPhone's biggest strength? Not all apps were made by Apple. In fact, almost all apps on the App Store are made by third party developers. While the iPhone keeps a tight rein on the platform, its platform of Apps might just be the best example of a horizontally scaling system that enabled the iPhone to be so much more than just a touch screen iPod, phone and an internet communicator.

Even in my own explorations of designing emerging technologies, I've tried to go beyond a singular appliance and build out systems or platforms that allow others to adapt it.

The Share Wagon Food Sharing System

The share wagon food sharing system is so simple and straight forward that it can be easily scaled and adopted by anyone willing to set it up in their apartment complex. The core of the idea is to share food with community. Such a system is easily adoptable.

The Thought Lamp isn't just an appliance for Math Problems

The thought lamp expands its actions and turns it into a physical and digital platform that allows anyone to build actions for it. Moving beyond just solving for students focus to a vision of augmented, ambient computing.

The Era of Techno-Centrism is Over

The success of the App Store brought with it the notion that software in your pocket could solve a multitude of problems, leading us into an era where tech took the center stage.

But as problems become more complex, and technology's impact grows, it's become increasingly clear that technology alone is not enough. It's only part of a bigger system that the problems and their solutions live in and when we start looking at those systems, tech's role in the solution space is hardly the center.

Designing emerging technology platforms with a systems lens, also marks a shift, where the understanding of the system, people and the platform  becomes central to the solution rather than the technology itself.

Further Reading:

1. H. W. J. Rittel, “On the Planning Crisis: Systems Analysis of the ‘First and Second Generations,’” Bedriftskonomen, vol. 8, pp. 390–396, 1972. [Online]. Available: http://www.nwlink.com/~donclark/agile/a%20-%20hold/Rittel-Planning_Crisis.pdfdougengelbart+1
   
2. S. L. Star and J. R. Griesemer, “Institutional Ecology, ‘Translations’ and Boundary Objects: Amateurs and Professionals in Berkeley’s Museum of Vertebrate Zoology, 1907–39,” Social Studies of Science, vol. 19, no. 3, pp. 387–420, 1989. https://www.jstor.org/stable/285080
   
3. D. H. Meadows, Thinking in Systems: A Primer. White River Junction, VT, USA: Chelsea Green Publishing, 2008.
   
4. D. C. Engelbart, “A research center for augmenting human intellect,” presented at the 1968 Fall Joint Computer Conference (commonly known as the “Mother of All Demos”), San Francisco, CA, USA, Dec. 9, 1968. [Online].https://www.youtube.com/watch?v=yJDv-zdhzMY
   
5. B. Victor, “Inventing on Principle,” lecture presented at the Canadian University Software Engineering Conference (CUSEC), Montreal, QC, Canada, Jan. 2012. [Online]. Available: https://www.youtube.com/watch?v=EGqwXt90ZqAcsis.pace+1
   
6. D. C. Engelbart, Augmenting Human Intellect: A Conceptual Framework. Menlo Park, CA, USA: Stanford Research Institute, Oct. 1962. [Online]. Available: https://www.dougengelbart.org/pubs/papers/scanned/Doug_Engelbart-AugmentingHumanIntellect.pdfarchive+1
   
7. A. Dunne and F. Raby, Speculative Everything: Design, Fiction, and Social Dreaming. Cambridge, MA, USA: MIT Press, 2013.

Early 20th century and 19th century furniture looked like this:

Then the Bauhaus Movement started. And everything became blander, more mass producable. So a coffee table began to look more like this:

This is what is happening to App Icons right now. Everything needs to be made for Light, Dark, and Tinted and Clear Modes. Everything needs to be ready to have multiple style variations, present brand logos, and be ready for ASO metrics. So an app icon is no longer one glyph that represents the app. It's a system of images that show up in different contexts in the app, and must present themselves as consistent.

So, instead of this:

We have this:

and this:

These icons maximise brand consistency (or rather sameness). Apple's Liquid Glass icons are Liquid Glass like, and they probably scale extremely well with all the different modes of icons that iOS and macOS support.

Same with Google and same with Adobe.

Undoubtedly, no one can say that the newer icons are "better" than the older ones. They're more consistent, yes, they "scale" across platforms and sizes while maintaining visual sameness (consistency as corporate designers will tell you).

They're meant for an assembly line of app icons that must be pushed out, in order to satisfy every thing but the want of a good visual design.

But here's the thing, the Bauhaus movement addressed the requirements of the industrial revolution, the need to scale manufacturing to meet the demands of many more people.

The industrialised app icons serve no one but managers and executives who conflate consistency with everything looking the same. Design leadership that cannot come with a design system that enables and celebrates creativity, and the harsh reality that companies care more about the "brand identity" than pushing something that's beautiful.

Most images online never get an alt-text description. Not because people don’t care about Accessibility, but because it takes time to look at a photo, think about a good description and write it down.

Shiuli (pronounced: She--ooh-lee) makes that process easy by helping people get started with the alt-text descriptions.

Simply pick an image, have Shiuli generate an alt-text description, edit it, and use it as alt-text in your images.

Alt-texts are great for Accessibility they help people using screen readers get a better description of the image. Making them even more accessible.

The best part about Shiuli? You never even need to open the app! You can use Shortcuts, Visual Intelligence or the Action Extension to get Image Descriptions from anywhere, making it a modular, versatile app that can run anywhere.

Below I go through the lexicographic, and design designs behind Shiuli:

About the name: "Shiuli"

Why give an alt-text generation tool that's so utilitarian, a name and icon like "Shiuli". And what does the word mean?

Shiuli (pronounced She-ooh-lee) is the hindi language name for a night flowering Jasmine (also known as Coral Jasmine) that's found blooming in Delhi and other parts of India during summers.

For an image description tool I wanted the name and icon to be evocative of the photos app, so that the connection between the two is clearly apparent. Which is why I chose the icon to be a flower.

Why the Shiuli flowers specifically? One fine day on a walk across the UC Berkeley campus this summer, I discovered the Shiuli flowers. Apparently for Berkeley, they bloom in summers.

Shiuli are the connection between two cities that I have been fortunate enough to call home. If I am to name an app after a flower, it'd have to be Shiuli.

Shiuli are the connection between two cities that I have been fortunate enough to call home. They grow both in Delhi in Berkeley. If I am to name an app after a flower, it'd have to be Shiuli.

The Icon Design

The icon design was a long iterative process. For the first couple of months, I simply called the app "Alt-Text". The icon then would be a generic image. At first I used the Hokusai's wave. but then quickly changed to something I clicked.

Briefly, I considered using Alty as the app's name, and Marigold (another plant that I love) as the app's symbol.

Once the name was set on Shiuli. The process then was to create an app icon that best represented the flower. At first I tried with the text on screen, since it's an app for alt-text, but eventually I found that flower itself was a much cleaner representation.

I tried different styles free form, cut, and accurate tracings, but eventually settled on something symmetrical and geometric.

An app icon or logo isn't merely a logo, it's brand identity. It must scale across different sizes, it must scale from simple line drawings, to glyphs, to skeuomorphic representations that look like an actual flower.  I did the same for Shiuli.

Design is Iteration

The development of the app started in April 2025. So why did it take me this long to release it? The reason is iteration.

While I had my first TestFlights running early in the development. I received a lot of feedback that shaped how the app looks, feels and works.

Shiuli Started as a Shortcuts extension, and barely had any UI in the app in its first build. Eventually through user feedback the app got better at the UI, providing options to edit the alt-text, and adding more actions along the way.

Post Liquid Glass, the UI changed drastically to embrace the specular highlights and glass effects in their entirety.

Action Centered Design

If you've been following my work over the last year, you must have realised that I have been championing for an action centered approach to app design.

The same's true for Shiuli. Shiuli from day one has been modular. It's an action-centered app through and through. Taking its primary action and turning it into an AppIntent, Shiuli builds UI on top of the App Intents and App Entities, rather than taking a phone app first approach.

The result is a product that works with cutting edge technologies like Visual Intelligence out of the box.

Information and Output: App Entities

All the information inside the app is governed by an App Entity, the App Entity. Each App Entity contains the image, and its alt-text description. All the information needed for Shiuli to present the UI.

Action: Generate Alt-Text for the image

The key action here is to generate alt-text, the action – an App Intent, takes an image as an input and provides the App Entity as the output.

The App Entity is then presented as UI across multiple modalities and Interfaces.

On Visual Intelligence The AppEntity, shows up in the UI with an Alt Text, on Shortcuts it turns into a data point for the next action, in the action extension it shows a version of the phone app's UI.

Only the UI changes across systems, the logic, and the data that powers the interface remains the same.

You can read more about Action Centered Design over here:

A short primer on Action-Centered Design

Tools follow action, just like form follows function. Action-Centered Design is a versatile framework that enables designers to design for actions first.

Pixel PostsVidit Bhargava

Supporting Cutting Edge Technologies

Shiuli on Visual Intelligence makes it easy to generate Alt-Text from anywhere. You just need to screenshot the screen, draw the area for whom you want the Alt-Text, and get it right there in Apple's Visual look up interface.

Because the app is so modular, it leads to interesting workflows such as this one where I press the Action Button and have it generate and copy the alt-text for whatever's on screen. Making the process of generating alt-text smooth, rather than cumbersome.

So that's Shiuli. It makes it easy to make images more accessible, and it's out now on the App Store.

Shiuli: Alt Text Generator App - App Store

Download Shiuli: Alt Text Generator by Squircle Apps LLP on the App Store. See screenshots, ratings and reviews, user tips and more games like Shiuli: Alt Text…

App StoreSquircle Apps LLP

Plus, It's not a subscription based utility, you just buy the app once and use it for lifetime.

This week Apple announced they'll be bringing a "Tinted" toggle for the Liquid Glass appearance. Adding a very thick blur to the UI if the person chooses that appearance. This is ofcourse in order to address Liquid glass' accessibility issues, especially when dealing with light backgrounds.

To me it came off as a surprise because Apple spent the summer tinkering around with the Liquid Glass UI, moving around the blur, saturation and depth settings to give it the almost right shape and then they've gone on to add this toggle. Clearly, they too see the Accessibility issues.

I think Apple can do better. I think they can take inspiration from their own pre-iOS 26 "Materials" design language and put them into Liquid Glass.

For Liquid Glass, IMHO Apple should provide a way to pick the type of Liquid Glass, Clear (i.e. the default Liquid Glass), Tinted (with greater blur and a color tint), and Frosted (a new type of material with even greater blur, but also a fun frost texture).

To illustrate why this is a better approach, I'd like to show some examples.

Spotlight Search

Creating a more legible search bar

Look at how the text below the search bar actually seeps into the search bar. Creating a very distracting layer of white text below the search bar, affecting the legibility of the text in the search bar.

In this search bar, a tinted glass effect ensures that the text below the search bar is not visible, while the colors still seep through. An added dark tint improves the legibility further. The blur isn't too deep. It's just slightly greater than the previous (clear) blur.

Apple Music Live Activity

Making the contrast better on a white-ish background

In this example we see how a lot of elements in this clear UI are less legible because of how the light refracts on the glass. Creating a confusing UI.

To solve this, I do three things.

1. Add a tinted glass background
2. Reduce the vibrancy on text and icons
3. Add a frosted glass layer to the seeker and volume control

This new UI works so much better. It's legible, and still shows the colors underneath the UI. Unlike Apple's approach of adding a diming layer underneath it.

Tab Bar

Creating a tab bar that works across different style of apps

This Tab Bar is good, but not the most legible tab bar. It also doesn't play with all the images. Especially because there is both text and symbols crammed into this little space.

A frosted tab bar on the other hand, retains the colors from the background, keeping the background light but at the same time adding a layer of frost, providing good contrast across the board to the tab UI.

I think Apple can solve their Accessibility woes with Liquid Glass by adding more materials to their design language. Liquid Glass is compelling and fun to use, but with any sort of transparency, it's very difficult to control how something looks and so it inevitably creates legibility issues in the UI.

Because Liquid Glass is not a traditional blurred background, fixing the issues needs the solutions to be a little more creative. I believe taking the approach to create a 3 tier system actually helps strengthen the material.

This summer I set out to redesign LookUp for Liquid Glass. It wasn't just a liquid glass update, it was going to be a major redesign, improving the app's layout, speed and interactions. It's a massive project, and I had only three months to complete it to get the app ready for the iOS 26 launch.

So I did what every prototyper does with big projects. Break them down into small, manageable chunks and make fake apps! Like every good prototype, the only thing real about them was the feature I was testing, everything else was scaffolding to aid the testing of the design.

What did each of these prototype apps contain? Here's a brief demo:

Sargent toolbar is a dummy app to test the result screen redesign, starting with the new toolbar controls.

The way it's designed, is that everything is a static image, except for the toolbars which are interactive and give a good sense of how the navigation of the new app flows.

Notice how janky the toolbar interaction is.

This updated interaction feels just right. Thanks to the fact that I was only testing the toolbars here. This was literally a 10 minute exploration with minimal code change. Fake apps are a massive win for rapid iteration.

Once I had the result screen locked in, I moved on to the new collections UI. New prototype. Only the collections screen was interactive, using dummy data.

The goal here was to test the interaction of tapping on a new collection and tapping across words. It serves no other purpose.

In fact, if you tapped on any other tabs they'd just not show anything. It's that straight forward.

The collector just served as a way to tinker and play with the collections UI, making the interaction robust and fast, everything else was scaffolding to give the illusion of a real app.

Root ranger was an interesting exploration. It's an app to test the origin maps feature. The feature itself originated from a very rough idea, i.e. what if we could map how a word has travelled across space and time.

For this one, I tried doing something I don't usually do, I "vibe-coded" my first exploration. But with more intention than asking ChatGPT to give me an app. I built out the 'map' of the action I wanted to support. Providing the information, the logic and the output user interface that I mocked up in five minutes.

The result was a usable prototype that let me explore the functionality for any word. Impressive for a 15 minute prototype. Fully interactive. But just an exploration of the feature. No other frills. No connections to the dictionary.

It served its purpose. It showed great proof of concept. Helped me iterate quickly and create a completely new feature in record time.

Apple's Foundation Models, introduced with iOS 26, enable apps to get artificial intelligence features through on-device Large Language Models, eliminating the need for using massive, power guzzling models for a small set of tasks.

Foundation models are immensely resourceful. They're also extremely tricky to work with, given how small the models are, they don't behave like ChatGPT or other LLMs, they need to be prompted in a very constrained environment and there are specific things they are good at.

So I setup a works-like test app to prototype new Foundation Model features on the one device I had that supports them. i.e. the iPad.

As you can see this is a total departure from other prototypes. It's a works-like exploration of ideas. "Designed" to give me maximum control over how the outputs show up, this is a bare-bones app where each tab works as a playground for different potential features.

I even built a prompt editor in there so I don't have to build the app again and again to get a new output.

Once the models were tested and designed to work exactly the way I wanted, I was able to put some of the features into the app. Like this Example Sentence quiz feature.

While on-device models are great, both on-device and server based models have one thing in common. They're incredibly slow by modern computing standards. And the amount of time or the exact steps they'd take to reach an output is unclear.

This calls for a new kind of a loading screen, one that's a little more playful than a circular progress bar. It needs to be as fun, and interactive as the liquid glass on iOS 26; it's so fun to play with, you almost loose track of time.

Like all other art pieces this little piece of interactive art, it starts as an abstract exploration and then converges into something meaningful. So, I built out a little testing playground to play with different styles.

The result was these stunning particle effects!

Here's a little secret. Every LookUp video I shared on the social media was not actually the LookUp app. It was Doppelgänger 26. An app that looks like LookUp, works exactly like LookUp, but isn't LookUp. It uses a series of charades, dummy data and convenient scaffolding that's identical to the strategies used in the other prototypes, to make it look like a ready product.

Why? It serves one purpose. To bring in all the research and prototyping I did all summer into LookUp. Bit by bit, I piece together the individual features I built out on fake apps. Gradually over the summer this app took the shape of LookUp; until it was ready to be LookUp, and then it replaced the old app.

With major design projects, it can be hard to get started, given the number of things required to be accomplished.

Breaking down the redesign into smaller, sizeable chunks and building it out bit by bit is the only logical thing to do. I hope this little sneak peak into the development process of LookUp was insightful.

You can try LookUp on Apple platforms here:

‎LookUp: English Dictionary

‎LookUp is an innovative English Dictionary app that combines a comprehensive English Dictionary with powerful vocabulary building tools. The app is perfect for English Learners and Word Enthusiasts who want to build their English Vocabulary. 1. Beautiful Word of the Day Illustrations ensure that y…

App StoreSquircle Apps LLP

With iOS 26, LookUp now has access to Apple’s on-device Large Language Models, making it easier to integrate AI into the app’s features when needed. LookUp’s AI vision has always been grounded in human goals and actions people perform to achieve them. Helping them learn better through thoughtful interventions.

LookUp 11.1 introduced Explainers and Pronunciation Tips. LookUp 12 goes one step further and adds new learning modes to the app, that help people build vocabulary in ways that enable them to actually build a better understanding of how to use the words in every day situations.

New Learning Modes

LookUp 12 uses the on-device Foundation Models to generate a set of example sentence quizzes. These quizzes, understand the person’s context and create a set of example sentences that enables people to better understand the word.

English Learners need a way to get better at using words in real world situations.

English Learners learn how to use a word by taking quizzes that quiz them on their understanding of a word, based on how it’s used in every day conversation.

Responsibly using AI

These learning modes aren’t just using foundation models to generate content. They are going through a methodologically crafted workflow to act on pre-defined guard rails to provide a holistic and reliable learning experience.

What that means is we use our own learning engine built on scientifically proven spaced repetition techniques and learning methods before ever giving AI any thing to generate. This ensures that the learning experience is one that’s reliable and not error prone.

Origin Maps

This is what a words origin looks like usually. I don’t know about you, but I am a visual person and this isn't a fulfilling or easy to understand experience for me.

English learners benefit when information is presented to them visually and one of the best ways of showing a word’s origins is to traverse its roots through space and time.

LookUp enables English learners to build their understanding of a word’s origins by visually interpreting its history on a map.

Responsibly using AI:

How do we use AI here? We use the on-device LLMs to transform the word’s etymology into a visual map that represents the word’s usage.

Again, LookUp isn’t just just asking the LLM to generate a map, it provides the Word, it’s etymology, and additional information that might help the LLM generate a comprehensive map.

Did you notice any sparkles here? Me neither. LookUp's AI features are rooted in real human needs and not in the urgency to use the shiniest – coolest new technology; and this is merely a first stab at Foundation Models.

As I play with them, I am sure I'll be able to create more useful features.

I've been working on redesigning and rebuilding LookUp from the ground up over the summer, and I just released the update this week to coincide with the launch of iOS 26; and just like every other year, this update too is a mix of Apple's announced technologies, often requested user feedback, and a step towards the direction I want to take the product in.

This year's update focuses on three key features: A complete redesign of the app, New Learning Modes, and Origin Maps. I will try to showcase all three of them in this portfolio of LookUp's changes.

Redesign

Anatomy of LookUp

The latest redesign for LookUp, simplifies the app into three key actions, i.e. Discovering new words, Collecting them, and Learning them. these actions mirror the entities responsible for each of these actions, i.e. words, a collection of words, and a learning box of a collection of words.

Discover

Language learners can discover new words either through the word of the day graphics or by searching for a word from the dictionary

Collect

Learners can collect / cluster the words together in different collections

Learn

Once collected, learners can then start taking quizzes and learning the words.

The simplified tab bar is a reflection of the key actions of the app

New Learning Modes

New learning modes people to learn more freely through through three  new learning modes. People can now learn through Interactive Quizzes. Example sentences, Conversational Questions.

Example Sentences presents contextually relevant and learner friendly example usage of the word, and quizzes the user on the meaning of a particular word in the sentence. This is a helpful way to remember how to use a word.

Conversational Questions offer a snippet from a conversation between two people. Based on that conversation the user is then asked to comment on a word that best describes the scenario. These questions dig one layer deeper than just rote learning a word's meaning. They check true understanding of the word.

The Learn with Example Sentences and Conversational Questions feature use AI in thoughtful and meaningful ways, i.e. helping people learn words, but also responsibly using AI to only generate content it can reliably generate.

The learning logic still uses scientifically proven frameworks such as the spaced repetition technique to create a learning set for the learners.

Origin Maps

Origin Maps is a fun new way to understand a word’s origins, which helps people to understand the word’s origins by tracing where it came from.

Origin maps are fun and informative in their own way. They help users understand a word's roots in a visual format. Also another cool use of the Foundation Models, transforming data from text to something entirely visual.

So this is LookUp 12, a major redesign for the app, that also includes meaningful updates that'll help language learners learn English in even more fun ways.

You can check the app out today on the App Store: https://apps.apple.com/in/app/lookup-english-dictionary/id872564448

“Eyeglasses are a good tool—you look at the world, not the eyeglasses” - “World is not a desktop”, Mark Weiser, Xerox PARC 1994

Case Study 1: Designing distraction-free computers for students to address the problem of smartphone addiction.

Designing a platform of ambient computers using Action Centered Design

The Thought Lamp is a Platform built for, and using Action-Centered Design

A versatile design framework

Action-Centered Design expands well-beyond a single platform. It's a versatile general design framework that can be used to design everything from apps to systems to interact art projects.

Further Reading

Designing Distraction Free Computers

Designing distraction free computers using an Action-Centric Design approach.

Pixel PostsVidit Bhargava

Beyond Sparkles and Chat

AI helps best when it stops taking the center-stage

Pixel PostsVidit Bhargava

Bringing Word of the Day by LookUp to Apple TV

How an action-centered-design framework and SwiftUI helped me build the Apple TV app for Word of the Day by LookUp in an afternoon’s worth of work.

Pixel PostsVidit Bhargava

Action Centered Design Framework

Full video of my Action Centered Design Framework talk at the ARCtic conference in Oulu; March 2025.

Pixel PostsVidit Bhargava

The way we interact with apps is changing, so should the way they are designed.

The current state of apps and the advent of Apple Intelligence makes it important to have an action centered design approach to designing apps. The phone is only a part of a much larger ecosystem.

Pixel PostsVidit Bhargava

Mark Weiser's The World is not a Desktop essay that forms the ideological basis of the design framework: https://dl.acm.org/doi/pdf/10.1145/174800.174801

One fine evening on my walk back to my apartment, I got a “Suggestion” by my phone; that it was a good time to check-in with my dad. The phone must have realized that I am walking back home, and for some reason thought that it’d be a good idea if I chatted with this contact who seems to live with me.

Sounds like a great idea, right? The phone is giving me proactive suggestions based on my “context”. Awesome! Except My dad passed away a little over three years ago. His phone was now handled by mom, who was at home at that moment, waiting for me to get back. Not only was the suggestion not helpful, it was triggering. I felt sad the entire walk back home.

The computer could never have that context on me. How would it know that my Dad is no more, how would it know that when I message “Dad” I am actually talking to my Mom? How much more context would it require to understand that this notification would be triggering for me?

But First What is Context? It’s the circumstances that build the setting for an event, or idea, a dynamic relationship between our thoughts and actions, and our surrounding elements—such as people, places, and object; that give our actions meaning.

For example: “I am going to get that” only contains meaning to another if the other person knows what that is. If I am in a library, and pointing towards a book, “that” would mean the book, In this case, the library, and my fingers form the context. If it’s a response to my brother asking if I got the yogurt from the grocery store yet, “that” would mean the yogurt.

We as humans see, think and do things under different contexts. Our words don’t live in isolation.

So when we use computers to perform actions that would fulfil our wants and needs, we must clearly delineate our intentions. Traditionally. The newly emergent idea in computing, asks, what if the computer had a good enough understanding of your “context” to perform those actions with minimal communication, or better yet, anticipate them and do the work for you before you even get to ask it.

If you’re to hear Mark Zuckerberg and much of the Silicon Valley today, they’d tell you something to the tunes of “Context is King” or “Context is absolutely essential” “AI needs context like you need food and water”; all of them are trying to say the same thing … The more context their AI has on a person, the more easily they’d be able to create a better computing platform. (Zuckerberg goes one step further. He wants to record everything you do, and everything you see; to create a digital twin of you). More Context = Better AI.

This is a fallacy. Because Context is dynamic, it’s relational i.e. it doesn’t live in isolation, and the more context you provide, the more it gets hungry for.

The intangible bits that form context

There’s the tangible personal context, that forms a major part of contextual computing’s discussion today; it’s your environment (location, weather, people you’re around with), your identity (information about you, your demographic, languages you speak, your preferences), your actions across (a trail of notes you make, your email, you calendar, contacts, and actions you take across different services).

But the tangible context is only part of the story, there’s also the intangible context that governs our actions. The social norms; practices and expectations that make us act a certain way, in certain conditions. We may say something but mean something else, our lived experiences form our thought process, they change how we act under different conditions; some people handle pressure situations differently from others, when I am acting under stress, that context matters in my interactions with the computer.

The Intangible Context is just as important as our tangible context in our usage of computers and other tools. It’s far more difficult for a personal computer to understand the intangible cues and context. So the picture of our context may always remain largely incomplete, even if we somehow gather as much context as possible.

The Paradox of Context

Coming back to the example of failed contextual computing that I shared in the beginning. More context should make this better right? But the more context I provide to such a suggestions based system, the more it needs to be accurate.

Each contextual element branches out into more questions, and the more context the system gets, the more it needs, to accurately display that suggestion. Mixing up on any one of those would result in an unsatisfying experience.

Moreover, this context is dynamic and changes based on location and what i am thinking about at that moment.

Sure, more context makes this suggestion helpful. It also makes it more prone to errors, it also makes it more prone to making a mistake.

Early Virtual Assistants

General purpose contextual computing burst into the mainstream when Apple introduced Siri in 2011. Early impressions of Siri were incredible, there was this voice assistant that suddenly understood natural language and did things! Amazing! Computing had a new paradigm.

Siri had a lot of context to begin with. But its contextual understanding of the world was also largely limited. So what’s the weather outside or should I wear a jacket would give me a good response, but if I asked a follow-up question like “share this info with my brother” it wouldn’t know what “this” meant.

People realised quickly that Siri’s understanding of real world context was largely limited. But they had no idea of what Siri did not know. So they’d ask it questions and every failure what add to the notion of it being dumb. That stands true even today.

Cut to 2022, ChatGPT leap-frogged the virtual assistant market, with something that had a much richer understanding of the world around it, and was able to respond to questions more clearly. The LLM powered chatbot had a lot more context than Siri; and was a much more impressive piece of technology.

Fewer errors yes, but the responsibility increases manifold. GPT is prone to lying (famously glossed over as Hallucinations), and doesn’t really have the mechanisms to verify if what it’s providing is true, and useful. But the UI does a bad job of telling that.

ChatGPT’s shortcomings are in its inability to situate its responses in the context of how it knows what it claims to know and its inability to gracefully fail when it doesn’t know something. Worse yet, the LLM is trained in a way to always try and find an answer, so it’s more likely to make up stuff when it doesn’t know something.

Moreover, it's hard to judge the extent of ChatGPT's capabilities, sometimes it'd surprise with a great answer, others it'd be frustratingly restricted.

Humane’s Ai Pin was based on the assumption that if GPT-style LLMs were provided more context, then voice would become a dominant and useful paradigm for computing. More Context + ChatGPT (and other LLMs) = Better Computer. Right? One of Ai Pin’s glaring deficiencies was people didn’t know what to ask, and when they did, they hit a wall really quickly.

Bad Contextual Computing Happens when there's a context mismatch

Bad Contextual Computing Happens not because there isn’t enough context but because there’s a lack of a shared understanding of context

Shared understanding of context

What’s missing from these examples is a shared understanding of context. When we communicate with our peers, there’s a shared understanding of context that’s established by the environment, our past relationship with one another and the circumstances we’re in.

There’s no such thing with a computer. It doesn’t know much about me from the beginning, and I don’t know what it knows about me. So if it doesn’t know something I expect it to know, I am disappointed (Siri), if it knows more than I expect it to know, I am alarmed, and suspicious (Why most people fear Instagram is listening on them).

Computers and Humans if they’re interacting with each other, need a shared understanding of context. i.e. I as a user should know what the computer knows about me, and the tasks I am trying to perform. It will never know everything so it must also provide me with easy ways to course-correct and nudge it in the right directions.

This is not a new concept. For as long as Human Computer Interaction has existed as a field, it has put forward the idea of shared mental models. The computer user interface must map the person’s mental model of how something works. Why? So that the two have a shared grounds for interaction.

Designing good contextual computing experiences

Key user Paradigms:

1. Bound the context: Contextual understanding of a system is not infinite, it’s finite. Designers must do a good job of clearly showing the bounds of a system’s context.
   
2. Be Clear: Clearly display what lead to a certain decision. This can mean showing what context a computer has on the user.
   
3. Ask Follow-Ups: When there’s not enough context, ask people to get relevant information before proceeding. Also important to disambiguate when conflicts arise. (3)
   
   Context isn’t static it’s a negotiation, between a person and the object they’re interacting with. Follow Ups and nudges make it easier to build an understanding of context. In our daily communication we constantly do this. Between our peers, we constantly negotiate context, by following up with questions.
   
4. Fail Gracefully: Everyone makes mistakes, computers do too. It’s important in those steps to provide clear pathways of resolving errors. Made an error, let people provide more context. Tell them it’s out of your knowledge bounds. Don’t make up information.

Finally, context as previously mentioned isn’t static. It’s dynamic and constantly being negotiated with the user. During the interaction process when the person clearly

Some examples of Good Contextual Computing

Conclusion

Hoarding large amounts of personal data, serves other interests of tech companies but it doesn’t solve the problem of creating computing that understands its users so they can achieve their goals easily, without having to communicate with a computer in verbose terms. Context is a paradoxical and dynamic entity, that’s more than just raw data, it begins with the tangible and intangible relationships people have with their environment but goes far beyond that.

TLDR; Companies are never gonna get “full context” on a person, especially not by capturing large amounts of personal data. Any company serious about contextual computing should stop fooling around with the notion of complete context.

Contextual computing experiences don’t fail because there’s not enough context. They fail because there’s not enough shared understanding of context. If the person using the computer knows and understands what the computer knows about them to perform an action, or in design lingo, “the UI’s mental models match the user’s mental models” the experiences are better.

The answer to contextual-computing is not more context. It’s precise, limited and mutually understood context, it’s about getting affordances and mental models of a UI right.

Essential Readings

(1) What we talk about when we talk about context - Paul Dourish - 2004
https://cseweb.ucsd.edu/~goguen/courses/275/dourish.pdf

(2) Intelligibility and Accountability: Human Considerations in Context-Aware Systems -- Victoria Bellotti,  Keith Edwards for Xerox Parc – 2001
https://faculty.cc.gatech.edu/~keith/pubs/hcij-context.pdf

(3) Boundary Objects — Star, Griesemer - 1989
https://courses.dubberly.com/design_theory_2017/04._a_Star_Griesemer_1989.pdf

Here's a quiz question. What does this sparkle mean? If you've been following technology news, it could mean any one of the following: It could mean AI Search, it could mean AI Image generation, it could mean a generic AI powered feature?

So Sparkles = AI? Right? But what is AI? The two letter word abbreviation doesn't really mean much. It's become synonymous with a ChatGPT style interface in the last couple of years, but a chat interface is not ideal in so many places. Moreover, Its definition is ambiguous because its use cases have been ambiguous too.

I can’t clearly understand how these two letters will make my life better, without more context. AI Image Generation? How is it better than normal image generation?

AI is not a feature, it's one of the many technologies that empower people to fulfill their wants and needs. It doesn't always have to be the center-stage, it doesn't always have to be in your face. The focus should always be on human needs.

Over the last couple of years I've developed a framework for designing with emerging technologies, i.e. the Action-Centered Design Framework. Here, I focus on the actions that people perform to fulfill their wants and needs and design tools that best help them perform those actions.

Here are some example of AI features, that go well beyond sparkle icons and chat interfaces and address real human needs.

AI features that go beyond chat

Not everything needs a chat interface and not every feature need to be called AI-thingamajig; features can be descriptive and go beyond traditional chat-interfaces.

Pronunciation Tips in LookUp

Over the years, non-native english learners who use LookUp have requested a better way to learn pronunciations. IPA notations and sometimes even the voice aren't enough, people want more tips to learn how to say certain words.

LLMs not only enable us to provide that information, they also enable us to provide contextual tips that understand the person's context i.e. in this case – their proficiency, native language and interests.

And that brings us to the pronunciation tips feature for LookUp. Pronunciation Tips provide an easy to understand phonetic pronunciation (sometimes even in the native language of the user if that's helpful) and helpful tips on what letters to stress on, and what to ignore.

Connotations in LookUp

Notice how LookUp's explainers use a shared-understanding of context always making it clear what the app knows about the person, and always making it clear when something is AI generated.

These AI generated explainers go beyond the basic definition, and help non-native english learners develop a better understand of how to use a word. One may understand the meaning of a word, but where to use it is a whole different story, helping them draw parallels to their own language helps a long way here.

Shiuli: Making Images more Accessible

My upcoming app Shiuli, enables people to share media with alt-text. It uses LLMs to generate an alt-text (text that describes the image in case it's not visible to internet users). The powerful vision models offer a great starting point for people to create alt-text for the images they're sharing online.

Often times, people even though well intentioned, struggle to get started with improving the accessibility of the work they share online; Shiuli bridges the gap, by meeting the people where they are and offering handy extensions and shortcuts to let them generate and copy alt-text for use easily.

Designing the next-gen of computing platforms

AI features can be used to design much more than just apps. They can be used to solve problems creatively.

Thought Lamp

The Thought Lamp is a smart lamp that serves a student's desk lamp until they pass it a piece of paper, it then transforms into a full-blown study companion projecting the interface on the desk, helping them perform the task right there; and then fading back into a traditional lamp when they're done.

This creates a distraction-free experience for students helping them stay connected to the work they're doing without being distracted by instagram or twitter notifications.

Want to solve a math-equation just pass a note with the equation on it and the thought lamp solves it, want to save class notes, just pass them to the lamp and it saves for your later use.

Designing Distraction Free Computers

Designing distraction free computers using an Action-Centric Design approach.

Pixel PostsVidit Bhargava

The thought lamp is great at solving math problems, taking notes, generating to-do lists for helping students stay on track with their tasks, but it's also a platform that helps developers create any action that'd help students stay on track.

Where's the AI? It's not naively solving math problems, It's the coordinator, helping pick the right action to respond to the user's query!

AI doesn’t need to take the center-stage

A lot of times, AI is not the lead actor. It’s a means to solve a bigger problem

Share Wagon

The Share Wagon is a community design system that works on community sharing of excess groceries in an apartment complex to reduce food waste generated by the large package sizes of grocery chain purchases. So if you only need 5 strawberries but your grocery store doesn't sell you less than 20, you can always share the excess 15 with the apartment complex.

The mobile app in the system acts as an inventory management system helping residents stay on track with whatever's available in the wagon. The app's UI shows how much and what item's available to pick from the Share Wagon.

Each time someone puts groceries on the share wagon cart; its weight sensors capture the weight of the grocery item and an overhead camera clicks a picture. Sending it to the app's servers where it uses AI to estimate the quantity of the food based on the food item identified from the captured image and the recorded weight.

AI isn't the lead actor here, it's not even the point of the system, in-fact the system works just as well without any tech; the proposed system is about sharing food amongst a community. But the app and AI help improve the quality of the system in a small but meaningful way.

Using AI in New Media Art

AI in art doesn't have to be generative. There are more ways in which we can use it that have a meaningful impact.

Agent Johnny: an interactive crime thriller on the idea of "manufactured water crisis"

In Agent Johnny, an interactive crime thriller. the viewer receives a phone-call mid way through the movie, where the lead character must seek advice, advice that only the viewer can provide.

Gradually through these interactions, the film's narrative changes, nudging the protagonist towards the film's conclusion. This way, the viewer becomes a change-maker in a high-stakes thriller. By the end of the movie, the viewer's not only entertained but also learns about problem of "manufactured water-crisis" a wicked problem with real world consequences.

The film doesn't any AI generated visuals, yet it uses AI in a way that empowers individuals to learn something new!

This is just a tiny glimpse of how AI can be powerful without hanging on to chat based interfaces and generic branding. It doesn't always have to be the loudest voice in the room, it can make meaningful impact in small, thoughtful contributions too.

Personally, I am cautiously optimistic about the new Jony Ive — Sam Altman partnership. Jony Ive is a successful designer with a lot of experience in designing amazing hardware for products that have redefined tech industry in many ways but he and his industrial design team are just one part of a much larger team that made those amazing products possible.

OpenAI partnering with Jony Ive is not a sure-shot way of designing the next big thing in computing. Yes, it’ll probably have the best industrial design that’s come out for a hardware product in a while. Yes it’ll have great AI models (that’s what OpenAI literally does). But will it be a product that people use? That remains to be seen.

If the two AI Hardware devices that released last year are any indication, there’s a lot of work to be done before anything can be successful in this field. Why? The problem with AI hardware is the fact that it’s called AI Hardware. This is a nonsensical term that doesn’t mean anything but Silicon Valley is hell bent on trying to give it meaning and form. The lack of creative naming is also a lack of product clarity in my humble opinion.

Who is the AI Hardware for?
Why do those people need it?
What does an AI hardware device do?
How is that better than the current solutions?

The answer to all of these questions is unclear, and it usually goes in the vain of AI hardware being a general purpose computing device, that will be useful for everyone, and will do the job that your phone does now but better because it is slathered in AI. Which to me is a recipe for disaster.

We’ve seen how it goes, General Magic, Humane Ai Pin, Rabbit R1, they’re all prime examples of this problem.

When you look at one of the most successful general computing devices of all time, the iPhone, even that had a clear user-base in mind and three clear use-cases (an iPod, a Phone, an Internet device) that made it stand out. It’s clear that general purpose doesn’t really mean general purpose, it still means a list of specific things that this new device would be incredibly better at, so much so that people will leave their existing solution and adopt it instead. This is not what I hear when I see AI hardware today.

What would I love to see? I’d argue it might actually make sense to make more uni-taskers again. Computing devices that do one task, but perform it really well (Like iPods, Kindle, Nintendo Switch). Not everything needs to be the next iPhone. It can start small and build from there on. Even the iPhone wasn’t as complex a computing device it is today.

At the very least I’d like to see a computing device with a clear user in mind, and a clear “this is how it impacts my life in a positive way, and this is why it’s better than that what people are using right now” pitch.