Wiki Contents

Deep Learning for Vision and Spatiotemporal AI

Project Description

This is an umbrella project for all deep learning-based research with special emphasis on vision centric and spatiotemporal AI applications. The current sub-projects are listed as follows.

1. Cancer Imaging & Medical Anomaly Detection

This project deals with using generative adversarial network (GAN) models for data augmentation via texture transfer and balancing. It is part of a broader project in medical informatics that seeks to apply self-supervised learning and differentiable computing (including deep neural networks) to open problems in medical imaging and image-based diagnosis.

2. Real-Time Face Recognition

In this ongoing study, we aim to identify the faces of livestock cattle by using deep learning to free farmers from using radio frequency identification (RFID) ear tags to identify cattle, a practice that is expensive due to sensor, ear tag, and installation labor costs. In the initial stage, we have been provided with videos of cattle taken by phone. Currently, we have about 1,000 videos; whose length varies from 4 seconds to 15 seconds. Images are also extracted from videos. One of the research objectives is to improve the data collection procedures, while another is to demonstrate recognition accuracy, precision, and recall scores comparable to the current state of the field as a proof of concept.

3. Autonomous Agents for Precision Agriculture

This project deals with using deep reinforcement learning (DRL) for policy learning in spatiotemporal domains such as precision agriculture tasks (pesticide and fertilizer application, image gathering) and autonomous navigation. An important part of this research is its focus on multi-agent DRL (MADRL). In recent years, single agent DRL algorithms have been shown to effectively learn desired policy and exhibit super-human performance. However, practical applications of DRL naturally involve more than one agent; for example, a multi-robot precision agriculture task (involving multiple autonomous agents) such as pesticide spray in a large field or removal of harmful invasive species from farmland, requires multiple robots which must learn to collaborate amongst each other and solve the challenge jointly, including path planning, navigation and task distribution. MADRL is also necessitated by the fact that the role played by each agent may be different and hence partitioning of the problem according to these roles can improve learning efficiency.

4. Automated Inspection & Engineering Anomaly Detection

This project is part of a spatiotemporal anomaly detection project that combines image processing, computer vision techniques for object detection and pattern recognition, geographic information systems (GIS) and aerial visual survey. Applications include analyzing unmanned aerial systems (UAS) data to identify and geotag systems such as streetlights and insulators that are inoperative. Research in machine learning and data mining focuses on a selection of relevant visual features and training data for capturing and labeling objects from flyover images. In this study, a deep learning based system will be developed which will automatically recognize damaged electrical insulators from images taken by a drone. The project will provide the development support needed to prototype and build a complete object detection and classification application. The system has several key features:

  1. Ability to locate insulators in images with a high success rate.
  2. Ability to identify damaged insulators with over 85% success rate.
  3. Design a web application interface that allows easy use of the system.
  4. Several ongoing research efforts focus on utilizing different types of generative models to combat class imbalance issues.

5. Automated Detection of Risky Tackle from Sports Video

The aim of this project is to classify safe and risky tackles from videos of American football practice. We have collected and annotated a data set of 108 videos by collaborating with experts from the Department of Kinesiology. The label 0 indicates the tackle is safe and 1 indicates the tackle is risky. Currently, we have 32 tackle videos marked as risky and 76 videos marked as safe. Our initial target is to frame the detectable region in space and time (spatiotemporal event detection), relate the "bounding box in space and time" to automatically-generated annotations, and link qualitative annotations (attention maps) to quantitative ones (analytic results such as estimated contact surface, force, angles). Also, we are trying to collect more data with a clean background.

6. Classification of Bee Species

This project is a collaboration with the Entomology Department at K-State which deals with automated identification of bee species from images. Our current focus is on Bumble bees. We have an in house dataset of 89,000 images of bumble bees, representing 36 species in North America. We have compared different convolutional neural network based image classification models. In the future, we are looking forward to improving the accuracy of bumble bee classification as well as adding more species of bees. You can try the BeeMachine web app.

Keywords

artificial intelligence, machine learning, deep reinforcement learning, information extraction, spatiotemporal pattern recognition

Methods

Current Team Members

  • Farrukh Ali (Autonomous Agents) - Ph.D. student, Computer Science, Kansas State University
  • Majed Alsadhan - Team Leader (Cancer Imaging & Medical Anomaly Detection) - Ph.D. candidate, Computer Science, Kansas State University
  • Nasik Muhammad Nafi (Autonomous Agents & Precision Agriculture) - Ph.D. student, Computer Science, Kansas State University
  • Ademola Okerinde (Automated Inspection & Engineering Anomaly Detection) - Ph.D. student, Computer Science, Kansas State University
  • Tom Theis, M.S. student, Computer Science, Kansas State University
  • Trevor Witt - Team Leader (UAS-Based Data Acquisition) - Data Analyst / UAS Flight Instructor, Kansas State University Polytechnic Campus
  • William H. Hsu - Professor - Computer Science, Kansas State University
  • Derek Christensen - M.S. Operations Research, Kansas State University
  • Emma Johnson - undergraduate student, Computer Science, Kansas State University

Alumni

  • Lei (Ray) Luo - Team Leader (Real-Time Face Recognition), formerly Automated Inspection & Engineering Anomaly Detection - Ph.D. candidate, Computer Science, Kansas State University
  • Vahid Behzadan - Ph.D. 2019, Computer Science, Kansas State University
  • Yihong Theis - Team Leader - M.S. 2019 Computer Science, Kansas State University
  • Rakshanda Reddy - Computer Science, Kansas State University
  • Carlos A. Aguirre - B.S. 2019 Computer Science, Developing Scholars Program and Undergraduate Research Programmer, Kansas State University
  • Luis Bobadilla - B.S. 2018, Computer Science, Kansas State University
  • Maria Fernanda De La Torre - B.S. 2018 Computer Science / 2019 Mathematics, Developing Scholars Program and Undergraduate Research Programmer, Computer Science, Kansas State University
  • Alice Lam - B.S. 2019 Computer Science, Kansas State University
  • Jordan Roth - B.S. 2019 Computer Science, Kansas State University

Data Sets

Black and Veatch - Data and Code on Docker

Trello Board

BeeMachine

Black and Veatch Project

New Vision Paper

Source Code

References

KDD Lab Publications

Last updated Wed 18 Mar 2020.

Last updated by pozegov on Jul 1, 2023