This is a glance at some of the projects I worked on, and the skills used.
NOAA Severe Weather Prediction
A 16 month machine learning project to detect damaging weather events, similar to that of NOAA's SPC (below), but globally
Components:
- Python scripts for gathering atmospheric weather data(NOAA) and saving to AWS s3 buckets
- Imputation of data that is available globally (NOAA is just USA)
- Inputting into machine learning pipeline for converting our 25+ variables into risk probabilities
- Several iterations of models, loss functions, feature selection
- Full stack pipeline that could automatically fetch, transform, predict, display, and show metrics and about the weather events.
Skills:
- Working with AWS (s3 buckets, EC2 instances, etc)
- Learning how to operate in a new domain (no prior experience in meteorology)
- Working with a variety of Machine Learning models
- XGBoost, Gradient Boosted Regressor, and Randomized Search CV
- Delivering and demonstrating results to a team as the sole person on a project
- Dealing with data lakes, using compression algorithms, and coding efficiently to meet the strict demands of "real time" output.
Global Building Footprints
The team was expanding their risk detection globally, and needed maps showing where buildings are located, along with stats on them. In some regions, this data is not available, and we needed to find ways to make our own.
Components:
- Filling in missing data to create a map of all buildings across the globe
- Extensively learned about data usage licenses and rights, such as creative commons, fair use, etc.
- Determined there were no commercially usable datasets of this kind (without limitations), so we built our own
- Took multiple approaches, LiDAR, satellite imagery + Machine Learning, accessing and gathering government datasets, etc.
- Used ArcGIS Pro for validation and cleaning
Skills:
- Working with several different map data formats
- Contacting government agencies in different countries to apply for access to their data
- Following strict data use guidelines and rules
- Retrieving, validating, transforming, and storing data from various sources, languages, and formats
- Performing all these tasks within ArcGIS Pro