a nyc based coding enthusiast, handyman, farmer at-heart, artist and future surfer 🏄‍♂️


Ocean view from Impossible Beach, Bali

My Mission

To master the process of software development, gain advanced proficiency in a few programming languages and use those skills and knowledge to build applications that:

  1. are novel solutions to everyday problems
  2. help others develop solutions
  3. make the world a better place than i found it
  4. brings people together

About Me

Hey there, welcome to my website! My name is Danny Lee, a NYC-based coding enthusiast who pivoted into software development in 2019. I've had a lot of past careers, including telco cable splicer, commercial real estate broker and small business owner, handyman, and a manager in different organizations. I also have a lot of hobbies including woodwork, acrylic and watercolor painting, letterpress printing, gardening, welding and a more recent interest in surfing.

Location

I'm a perennial New Yorker, born in, raised and lived most of my whole life in the greatest city in the world. Its an amazing, diverse and constantly changing landscape that is always renewing and rebuilding itself. Recently, nyc's taken a sharp shift towards tech and startups, which is incredibly exciting.

What do you love about software development?

I've recently shifted into software development and I love it! What do I love about it? Well first, although its difficult to pull off, the possibility of working remotely was that last push I needed to take the plunge. Bali, here I come! 🏄🏻‍♂️🌊 Next, there is an endless supply of knowledge to feed my love of learning and practicing new skills. Programming is a challenging, yet highly accessible field. The knowledge is readily available in different forms (books, documentation, videos, podcasts, et cetera). There's just so much of it, which is another positive reason for me. I love learning. Thirdly, I see potential for building passive income streams from apps and websites, that just need some time, an entrepreneurial spirit and some cash to get off the ground. And lastly, but perhaps, most importantly the challenges of programming aligns exceedingly well with my creative problem solving skills and there is an endless supply of knowledge to feed my love of learning and practicing new skills. Dream, explore, create, review, rebuild. Its how I approach my work, my hobbies and interests, its how I live.

As a handyman at Taskrabbit, I had the chance to take on all kinds of odd jobs and come up with solutions and bring them to fruition. Its at that time, that I truly learned my passions are in finding and building creative solutions for peoples' real world problems. I'm actually really proud of the reviews and kudos I received from my many clients during that time (link: https://tr.co/dhl), because I didn't grow up as a handy person. I also wasn't formally trained in the field, though 5 years as a telco cable splicer taught me a lot.

Handywork was something I always wanted to do growing up (having been a huge fan of This Old House and Bob Vila). After deciding to try my hand at it, and jump into the handyman field, I learned through reading lots of books and websites, YouTube videos, scouring handywork forums and reddit, and last but not least--the way I learn best--hands-on experience. It was scary at first, and it had a challenging learning curve, but I believe I can say, humbly, that I mastered handywork. But, I couldn't see doing it for another 15 or 20 years. I needed a new challenge and I think I've found it in software development.

We'll see how this goes! 😊

Quote of the moment

“Extroverts are more likely to take a quick-and-dirty approach to problem-solving, trading accuracy for speed, making increasing numbers of mistakes as they go, and abandoning ship altogether when the problem seems too difficult or frustrating. Introverts think before they act, digest information thoroughly, stay on task longer, give up less easily, and work more accurately. Introverts and extroverts also direct their attention differently: if you leave them to their own devices, the introverts tend to sit around wondering about things, imagining things, recalling events from their past, and making plans for the future. The extroverts are more likely to focus on what's happening around them. It's as if extroverts are seeing "what is" while their introverted peers are asking "what if.” ― Susan Cain, Quiet: The Power of Introverts in a World That Can't Stop Talking

Portfolio Projects

Twitter Geographic Trend Analysis


Team 6 Project 2
Using Big Data technologies, and Twitter's Trend API endpoint, project 2 consists of retrieving responses from Twitter with the Spark Structured Streaming API at 10 minute intervals, transforming the response with Spark by converting a single object with a location value and an array of 50 trends into 50 row entries and writing them to CSV format The output is then analyzed using Spark SQL to extract trend patterns to find the trends average rank, highest rank, hours trending, number of tweeets about the trend and top trends. The output of the project include Spark jarfiles, both pre and post processed datasets and documentation to repeatable and used on larger datasets and datasets that streamed on an ongoing basis. The questions examined follow: Can we identify patterns in trending topics being popular in one location and then moving to others? If so, can we identify separate regions for these patterns? (ex.: North American patterns, East Asian patterns, etc.) What about global scale patterns for specific events? ex.: Burning of Notre Dame-how did this topic travel according to twitter usage?

Roles / Responsibilities
  • Utilized Postman and it's Collection Runner functionality to script the automated retrieval of Twitter Trend API events.
  • Designed a Scala Spark Runner program to input raw JSON response files for processing.
  • Extracted Twitter's JSON Trend Api response and fitted it to a Spark schema.
  • Transformed the data from one object with a Location and fifty Trend topics to fifty objects with a Trend topic at that location.
  • Exported data into the local file system as CSV.
  • Destructured SparkSQL query function into separate Scala modules and packaged them as individual JAR files.
  • Tested functionality of SparkSQL JAR files in Google DataProc.
  • Produced and reported on analysis results.
  • Implemented Scala, Spark on Windows, and Ubuntu Linux.
  • Source control and versioning handled using Git and GitHub.
Environment / Technologies Scala, sbt, YARN, Parquet, Spark SQL, Spark Streaming, Spark, Google DataProc, Google Cloud Storage

COVID Dataset Analysis


Project involves a series of data analysis questions about the 2020 COVID pandemic. Analysis is performed using Spark applications (predominantly Spark SQL) on static and streaming data sources, making use of a Spark cluster and a Kafka cluster running on Amazon's cloud. Analysis questions and goals follow: Which Regions handled COVID-19 the best assuming our metrics are change in GDP by percentage and COVID-19 infection rate per capita. (Jan 1 2020 - Oct 31 2020)? Find the top 5 pairs of countries that share a land border and have the highest discrepancy in covid-19 infection rate per capita. Additionally find the top 5 landlocked countries that have the highest covid-19 infection rate per capita. Live update by Region of current relevant totals from COVID-19 data. Is the trend of the global COVID-19 discussion going up or down? Do spikes in infection rates of the 5-30 age range affect the volume of discussion? When was COVID-19 being discussed the most? What percentage of countries have an increasing COVID-19 Infection rate? What are the hashtags used to describe COVID-19 by Region (e.g. #covid, #COVID-19, #Coronavirus, #NovelCoronavirus)? Additionally what are the top 10 commonly used hashtags used alongside COVID hashtags? Is there a significant relationship between a Region’s cumulative GDP and Infection Rate per capita? What is the average amount of time it took for each region to reach its first peak in infection rate per capita?

Roles / Responsibilities
  • Point person for Docker, Kubernetes, Amazon S3 and Elastic Stack (Beats, LogStash, ElasticSearch, Kibana).
  • Designed a Scala Spark Runner program to input raw JSON response files for processing.
  • Write and maintain documentation for Docker, Kubernetes, Running Spark jobs on Kubernetes (Amazon EKS). [github directory]
  • Sourced documents, sample files, and wrote documentation for ELK, Spark, Git, DataProc, and Kubernetes/Docker.
  • Spark-Submit jobs to Spark cluster running on AWS Elastic Kubernetes Service (EKS)
  • Ran Spark jobs on Google DataProc using Google Cloud Storage.
  • Utilized Amazon S3 to store Datasets and Spark JARs
  • Ran Spark jobs on Amazon EMR in Cluster mode as well as Step Execution mode.
  • Collaborated using Git on Github to manage source control between multiple teams.
Environment / Technologies Scala, sbt, Parquet, Spark SQL, Spark Streaming, Spark, AWS S3/EC2/EMR, Google Cloud Storage/DataProc, git, Kubernetes, Docker, Elastic Stack



TTP app screenshot

TTP Stock Trade


A stock purchase app, built for Stage 2 of the NYC TTP application.
DevTeamer app screenshot

DevTeam - Dev Team Building


A web app to design teams with Agile roles.


Websites I've worked on recently (2020)


Past website work (2011 - 2018)

Industry Knowledge
  • Big Data Development
  • Full-stack Software Development
  • Handywork, Maintenance and Repair
  • Computer HW/SW Installation and Troubleshooting
  • Video/Photo Editing
  • Office administration
  • Commercial RE Sales and Marketing
  • Property Research and Analysis
  • Telecom Cable Troubleshooting and Splicing
Tools and Technologies
  • Scala
  • Hadoop
  • Hive
  • Spark
  • AWS EMR, Google DataProc
  • IntelliJ
  • sbt
  • AWS Certified Cloud Practitioner
  • Javascript with React
  • Ruby / Ruby on Rails
  • HTML/CSS
  • REST APIs
  • Git/Github VCS
  • Heroku/Netlify
  • FTP/Telnet
  • Wordpress
  • MS Office / Google Apps
  • Acrobat, Photoshop, Premiere
  • Most Hand and Power tools
  • Some Woodworking tools :)
  • Flux Core Welding
Hobbies and Interests
  • Acrylic and watercolor painting
  • Video art and mashups
  • Woodworking and welding
  • Gardening (indoor and outdoor, propagation)
  • Creative Writing
  • Karaoke
  • and, soon, Surfing!

Presentations
General
Technical
Kurt Vonnegut Cat's Cradle dust jacket (1963)
Creative Karass

Last updated: 5/26/2020
[image source, The Books of Bokonon]

[Myself at 38 w32nd Street with Canvas]
Art Galleries:

Last updated: 5/27/2020
[personal photo]

[South Korean Flag] 한국 물건 - Korean related stuff
[image of korean characters]
Language

Last updated: 5/26/2020
[image source]

[Kim Hong-do, A Cat and a Butterfly painting]
Art

Last updated: 5/26/2020
[image source]

[Seoul Food Shop 1890-1923]
Historical

Last updated: 5/26/2020
[image source]


[Indonesian flag] Bali

| | [weather image]

[image of sunset and temple in Bali]
General Info

Last updated: 5/27/2020
[personal photo]

[I love bali sign]
Cities and Towns

Last updated: 5/26/2020
[personal photo]

Graffiti on wall
Language

Last updated: 5/26/2020
[personal photo]

[Legs crossed next to a pool]
Digital Nomads/Freelance

Last updated: 5/27/2020
[personal photo]