Generative recipe recommender using on-hand ingredients, with continual learning & selective forgetting.
College students need a quick and easy way to decide what to cook for dinner using the ingredients they have on hand.
Problem
Users
Living on their own (away from their family and off campus) for the first time
Limited time and budget
Want to make use of the food they already have in their pantry.
Do not have extensive culinary knowledge (may not have a mental list of go-to dishes, and may not be able to “see” a dish given a list of ingredients)
Assumptions
Personalized Recipe Recommender
Food in pantry
User allergies
User preferences (likes/dislikes)
Transformer
+
RecSys
List of
recommended recipes
User has basic appliances (microwave, stove, oven) and basic cookware (utensils, pots, pans)
As a further extension, we could make available appliances and cookware editable by user, which could make the application useful for students living in the dorms as well
Design Process
We focused on the transparency and explain-ability for the machine learning recommendations in the UI.
Soundness: make sense because labels relate to recipes (e.g. “chicken expires soon” → recipes include chicken)
Proximity: located directly above suggested recipes
Personalization: specific to each user (“your _____”, “you like”)
Designing for Failure and Human-in-the-Loop Design
A before & after application usage survey was administered to 1,489 users we found; a 36% increase in cooking fun, a 70% increase in cooking creativity, 31% reduction in food waste, 10% increase in time spent, 20% decrease in money spent, 45% increase in finding new things to eat/make
We found that users desired a more simple UX. Despite our stripped down interface, they wanted something slightly more straight forward. We are considering hiding the pantry in the settings and approaching the correction for the recommender system in a less involved way, which would instead use data like time spent on page, and how quickly they clicked away from a recipe.
We make it apparent that the data collected and used by the application stays on the user’s device and does not, at any point, stream to our servers. The trade off to this privacy protection is that our application is a large download.
We are also explicit on how we get the data we do use for application improvements (only voluntarily, if users choose to respond).
Forgetting & Deleting
We give users the ability to edit their current interest buckets or to clear their data completely. Allowing the AI to forget individual user preferences enables the user to control their digital image if they are making a lifestyle change (for example, beginning to follow a vegan diet). So, a user’s ability to edit their buckets limits the reinforcement of undesirable or irrelevant recommendations.
Allowing users to delete all of their data enables them to protect their own privacy, in the case that they no longer want to use the app.
*Shou. Wired, 2019; https://www.wired.com/story/the-next-big-privacy-hurdle-teaching-ai-to-forget/
Ability to Correct AI Error
Users can manually edit AI-generated online pantry entries if they are incorrect. This way, the inputs to the model will be a correct representation of the ingredients a user has.
Providing Choice Improves Model
User’s decisions (which recipes they select/do not select to cook, whether they like/dislike a recipe after cooking it) give them agency and are used to improve the model’s inferences about their food preferences.
Generating text with OpenAI API
Complexity: appropriately complex explanation is provided based on context (explanations on the main page are succinct, while those on this page provide more information)
Context-full: explains how the model functions generally, as opposed to only in a specific case
Kacper Sokol and Peter Flach. 2020. Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approaches. In Conference on Fairness, Accountability, and Transparency (FAT* ’20), January 27–30, 2020, Barcelona, Spain. ACM, New York, NY, USA, 12 pages. https://doi.org/10. 1145/3351095.3372870
Data Handling
Privacy
iOS
Building using Swift & Swift UI, we use OpenAI’s API to call our tuned model to generate user specific recipes.
Generating images with OpenAI API
Building using Swift & Swift UI, we use OpenAI’s API to call our tuned model to generate images of the defined dish corresponding to the generated recipes.
Impact
Improvments