* Existing information/knowledge is distributed, scattered and simply not enough.
* Daily waste is often messy.
* Every county, state, country, has slightly different recycling regulation.
* Waste is only the tip of the iceberg of environmental sustainability issue.
* Sustainability is complex, involving social, economics, and environmental dimensions.
* Knowing is one thing, doing is another. 1. Limited and biased waste datasets.
* Traditional image augmentation and segmentation may have limits, requires innovative methods to improve imbalance training set.
2. Limited theoretical grounded learning tools in this domain.
* Fast changing pace in waste management.
* Informal learning of complex social problems (location, resource, regulation).
* Insufficiency of waste management knowledge.
- How to learn & re-learn?
- How to build sustainable behaviors?
- How to transform sustainablity awareness to contribution?
3. Challenges in visualizing and contextualizing carbon consumption & emission.
* Invisible, long-term consequential effect.
* EPA’s Carbon footprint calculator focuses on lump sum and metrics centered events, such as commute, transportation, household, etc. but consumer-based sustainable behavioral based carbon emission conversions.
* Embedded carbon is hard to capture or to visualize.
4. Challenges in finding, receiving, subscribing updated information, resources, and eco-tips.
* How can modern AI (LLM, ChatGPT, Generative AI...) help in addressing this challenge?
5. How do we bridge understanding (knowing) to actually doing (performing) sustainable actions?
* Help people navigate the complex waste management field.
* Provide emerging technologies to facilitate (lifelong, informal, sustainable) learning.
* Support sustainable practices and healthy behaviors.
* Raise sustainability awareness.Several datasets are adopted to train the object detection model, including TrashNet, Open Image Dataset (OIDv6, for straw & plastic bags), TACO, and our self-built dataset (for paper and plastic objects). Additional cleaning and processing were implemented to unify the format of annotations, which resulted in 4,835 distinct images with corresponding labels. We also randomly selected some of the images for augmentation and generated 8,596 samples in our dataset. The model was trained under the EfficientDet and then converted to a tflite model for real-time object detection in the mobile platform.
Datasets and model engineering lead to another research question documented in the later section.
While there ares everal existing published carbon calculators to help compute the equivalent amount of carbon dioxide (CO2) emissions, many of them are bundled with household or travel with a variety of input and configurations. i.e. https://www.carbonfootprint.com/calculator.aspx
Our solution utilizes the waste detection to mimic and simplify the abstract measurements of the amount of waste to trash and projecting an estimation of the carbon emissions. Associating the detected waste to the carbon emissions calculation avoids tedious steps of manual input and permits persistent carbon emission tracing.
In addition to the instant feedback of the detected waste & appropriate bin to dispose, we explore the immersive technology - AR to extend the information space. It provides more engaging interactions and three-dimensional space to display potential recommended information.
Forthcoming...
We recognize the importance of learning content and the challenges to involve users in using the application, a more streamlined UI has to be designed to compliment and maximize all the wonderful sustainability techs that we have built.
Sun., Q., Hsiao., I-H. & Chien., S-Y. (2023) Immersive Educational Recycling Assistant (ERA): Learning Waste Sorting in Augmented Reality, IEEE, the 9th International Conference of the Immersive Learning Research Network.
Sun, Q., Hsiao, I. H., & Chien, S. Y. (2023, July). Immersive Educational Technology for Waste Management Learning: A Study of Waste Detection and Feedback Delivery in Augmented Reality. In International Conference on Human-Computer Interaction (pp. 509-515). Cham: Springer Nature Switzerland.
Internet crawled images can be noisy. (i.e. an image labeled as straw often is not just a straw, it usually comes with a cup, a hand and a place, etc.)
Small objects can be hard to train.
In our previously deployed model (assembled upon open data sets, and some additions from our efforts), there is a total 55043 images, 24 categories.
Approaches to address imbalanced data
### Approaches to address imbalanced data
There are still problems...
- Cropping, blurring...It doesn’t guarantee imbalance data set issue. (underrepresented categories).
- Ensemble methods (AdaBoost, SMOTE) to learn from misclassification and reduce bias or to generate minority classes to rebalance the dataset. These models may be sensitive to noise and outliers.
Detail forthcoming...
Existing Open Datasets
| Data | Images & Annotations | Citation & Source |
|---|---|---|
| TACO | waste in the wild with 1,500 images and 4,700 annotations | P. F. Proenc ̧a and P. Simo ̃es, “Taco: Trash annotations in context for litter detection,” arXiv preprint arXiv:2003.06975, 2020. |
| TrashCan | 7,212 annotated images of undersea wastes | J. Hong, M. Fulton, and J. Sattar, “Trashcan: A semantically-segmented dataset towards visual detection of marine debris,” arXiv preprint arXiv:2007.08097, 2020. |
| TrashICRA19 | ||
| MJU-Waste | ||
| TrashNet |
Sun., Q. & Hsiao., I-H. (2023) Effective Use of generative AI for Environmental Sustainability: A study of combating small and imbalanced datasets in engineering waste classification models (submitted)
We applied Theory of Planned Behavior (TPB) to model the users’ behavior intention in waste management. The underlying assumption is that we all want to practice sustainable behaviors for a better tomorrow (i.e. proper waste sorting; recycle; reduce trash; etc.), but there is a gap between intention and actual behavior.
There are significant relations between Self-efficacy and Literacy & Litercy and Attitude.
Waste management knowledge (literacy) indeed plays a crucial role in the mix of self-believing one can be part of the environmental crusade.
Sun, Q., Chien, S. Y., & Hsiao, I. H. (2023, July). Theory of Planned Behavior Modeled Educational Technology for Waste Management Learning. In 2023 IEEE International Conference on Advanced Learning Technologies (ICALT) (pp. 74-78). IEEE.
Waste Genie is a web-based application that ensembles a suite of AI, AR & Social technologies that we've researched above. It provides users the most updated waste management content and support. Waste Genie is designed to be THE go-to place and tool to get well-informed for sustainability resources, eco-tips, waste sorting feedback, etc. to adapt to our forever-growing-complex and sacred environment.
To test the app, try it here: https://era.sqmlab.com/
More detail is forthcoming...
Read & interact first, publish and contribute later.
Sun, Q., & Hsiao, I. H. (2023, October). Waste Genie: Learning Environmental Sustainability from Waste Sorting and Interactive Feedback. In 2023 IEEE Global Humanitarian Technology Conference (GHTC) (pp. 310-317). IEEE.
Sun., Q. & Hsiao., I-H. (2023) Waste Genie: A Web-Based Educational Technology for Sustainable Waste Management, In Poster Proceedings of the Annual ACM Symposium on User Interface Software and Technology (UIST 2023)
Sun., Q. & Hsiao., I-H. (2025) Waste Genie: Emerging Technology Support and Interactive Feedback to Enhance Sustainable Waste Management Learning, Journal of Interactive Learning Environments, 1-18.
RQ1: How do we create meaningful sustainability learning content, and sustainably?
RQ2: What are the alternative UI to interact with AI agent? (considering the waste mgt field is so immense, the vocabulary and pre-knowledge may be limmtied.)
RQ3: How do we engineer a location-based legislation-guided LLM (llLLM) to assist educational content generation in WG? We are experimenting and evaluating Expert, Crowdsourcing and AI approaches to create content, specifically, how LLM can be capitalized in creating sustainability learning content, and is the method sustainable?
We experimented the open source LLM applications, chatGPT 3.5 Turbo, meta's LLaMA2, etc, the results are either not correct or too generic, and most importantly, not location specific. The straighforward solution is to integrate all the legal documents (legislative regulations about waste) into the LLM, however, it also indicates a severe hulucination of the model.
We also experimented engineering Environmental Legislative-guided LLM to help extract relevant information. Please refer to our IEEE CAI'25 & ACM COMPASS'25 papers.
Environmental Legislative-guided LLM for Content generation and evaluation pipeline.
Sun., Q., Chien., S-Y. & Hsiao., I-H.(2024) Learning Waste Management from Interactive Quizzes and Adaptive GPT-guided Feedback, EDM workshop on Leveraging Large Language Models for Next Generation Educational Technologies, the 17th International Conference of Educational Data Mining.
Sun., Q. & Hsiao., I-H.(2024) An AI-infused Educational Technology to Cultivate Self-directed Learning in Sustainable Waste Management, GoodIT '24: Proceedings of the 2024 International Conference on Information Technology for Social Good.
Sun., Q. & Hsiao., I-H.(2025) Engineering Legislative-guided LLM to Support Waste Management Learning, IEEE CAI'25.
Sun., Q. & Hsiao., I-H.(2025) AI-infused Educational Technology for Continuous Waste Management Learning: an Environmental Legislative-guided LLM enhanced approach, ACM SIGCAS&SIGCHI COMPASS.
Some memes are discovered in our user generated content. We later explore how AI can help generating engaging memes. Check our pubs.
More detail is forthcoming...
We create an AI assistant that helps users express and develop ideas about sustainable practices and use a commentary interface to collaboratively generate Reduce or Reuse ideas.
Sun., Q. & Hsiao., I-H.(2025) Human-AI Collaborated Ideation for Learning Reduce & Reuse Waste, ACM UMAP LBR. The 33rd ACM International Conference on User Modeling, Adaptation and Personalization. [slide]
We experimented many methods and off-the-shelf LLMs to generate educational content for WG. No matter static or interactive posts in WG, we need massive amount of organized texts, engaging imagery, clear background, and most importantly, the meta data for the post (i.e. waste sorting labels, regulations, etc.)
Couple studies are under way, i.e. the effects of GPT-feedback in learning.
Stay tuned...
Nickel, R., & Hsiao, S. (2024). Creative Collaborator: AI-facilitated UI for Creating Engaging and Insightful Memes. Human Factors in Design, Engineering, and Computing, 159(159).
One of our talented SCU students, Vladimir Ceban, designed Waste Genie logos.
We are very pleased and proud to announce some of designs here.
Concentrated mini-practices with AI-infused feedback (GPT-3.5) efficiently improved users' waste sorting accuracy. vs. practice on their own.
Landfill category is the main category that users struggle to sort!
*We have more games on deck, stay tuned...
Sun, Q., & Hsiao, I. H. (2024, October). Serious Practices for Interactive Waste Sorting Mini-game. In Joint International Conference on Serious Games (pp. 134-141). Cham: Springer Nature Switzerland.
As our team explore more and more about EdTech to support sustainability, we face one of the critical challenge, which is the gap
between knowing and doing. So we begin to research methods to support sustainable actions.
stay tuned...
Qiming Sun & Dr. Sharon Hsiao
We presented WG work in 2023 ACM UIST conf.