100 Days of Code - Self Challenge
We started building this website by taking the "100 Days of Code Challenge" and we must credit some our perseverance to the challenge. Since the challenge is in part (see rules below) to code for 100 days straight, essentially without a break. While we have met that part of the challenge, it is not the best practice for one's health. So the usual caveat applies - don't try this at home. To be fair is is a common situation for startups.
Current Status of the Challenge
We are now more than a year past the end of 100 day challenge (Feb. 14, 2023). We knew from the beginning that the resulting code would be closer to a good start than a "full platform". Given another year of development we are beginning to understand the true size of the overall project. Like in most software development projects, you learn by doing. We have been learning a lot.
Rather than focus on the yet-to-dos, this post reviews what has been done.
My original post showed three main development areas: the search engine, the reader app and the blogger app. I- The Search Engine: Basic blog site crawl is working and can be fired from either the Admin app or the Blogger app.
Features:
- Searches a blog site for postings.
- Selects the best image for the posting.
- Extracts the title, description and content.
- Ranks and stores the posting.
- The Blogger App: Basic blogger app. Blogger can login, verify their blog site, request indexing and remove any "posts" that are not posts. Indexing takes 1-2 minutes because it is polite.
- The Reader/Public App: The public side (sigin not required) has browse by category and search capibilty.
Challenge on! I am going to try the 100 Days of Code challenge in building a blog discovery site. We are not talking about a "learn to code" diary here. We are talking a killer web app as powerful as the big platforms like Facebook/Instagram/Pintrest/Twitter, but focused on the blog community. It will be a content (blog) discovery and sharing platform having a mix of search, browse and some social functions.
The Rules
The rules are simple:- Code for at least 1 hour every day for 100 days,
- Tweet progress daily,
- Reach out to two other people doing the challenge daily.
I will try and exceed the 1hr a day minimum, as I think reporting back on 1 hour of development work is not interesting and too much overhead.
Easier said than done, but that is the point – its a challenge!
Challenge Goal
The goal of doing the challenge is to produce a “full blown” blog discovery site (web app). The quickest comparison would be the features of Google Search + Pintrest. So no pressure.My Background
I am a software designer/developer with decades of experience covering all levels. I am well aware I still have a lot to learn and will be tackling some development on this project, like AI that I have little experience.Tech & Tools
I will be using what you would call a “Low Code” development tool, that I developed myself. It is not “release quality”, but it will enhance my productivity. I call it a MetaCode IDE and it differs from a Low/No Code IDE in that programmers can "feed the beast" by providing hand coded source code that can be used over and over.Development Plan
While I have a general idea of the functionality (love child of Google Search and Pinterest) I will be actively soliciting direction from blog readers and writers. That will influence what is developed and the order in which it is developed. You can participate and submit suggestions, by following @whosagoodblog on twitter. At this point I am going to divide the 100 days into:- 30 days development focused on search and indexing
- 30 days development focused on the blogger experience
- 30 days development focused on the reader experience
- 10 days of fix & polish
Adhoc Reports - Reverse Chron
Day 75
Clearly I have not kept up with daily reports, but I have kept up with development. Although there have been a few bumps on way progress has been steady, if slower than hoped. Today I opened the site to bloggers for a preview. The "Reader/public" view looks reasonable and readers can browse and search for blogs. Bloggers can sign-in and submit their sites to be verified and crawled. Next up, blogging about the site. Then updating the admin to manage internal feedback. Then responsive design.Days 17-18
Now reading nested sitemaps.Days 15-16
Building a blog discovery site Continued to code crawling the blog sites to find and index the posts. Time: 4hrs x2Day 14
Building a blog discovery site Did some refactoring. Now reading and parsing the site map. Started code to crawl the blog sites to find and index the posts. Time: ~4 hrsDay 13
Did some refactoring. Worked on reading and parsing the site map. Started code to crawl the blog sites to find and index the posts.Time: ~4 hrs
Day 12
Now scraping external (live on the interweb) blog home pages extracting: Title, description, keywords and images. Challenge was picking out an image (ongoing). To do: Allow blogger to select their preferred image.Time: ~6 hrs
Days 11
Now scraping live blog home pages extracting: Title, description, keywords and images. Needs more development, its messy out there.Time: ~6 hrs each day
This is an unlinked page
Day 10
Now scraping the blog home (test) page extracting: Title, description, keywords and images.Time: ~6 hrs
Day 9
Scraping web sites is like scraping paint. Takes Time.Time: ~5 hrs
Day 7
No rest, but I did take a step back and re-factor a couple of Blogger modules. Removed a switch and a half dozen if statements. Happy with the re-factored code.Time: ~5 hrs
Day 8
Started work on scraping a blog. Time: ~3 hrsDays 4 & 5
Have the adding a blog url and verifying the ownership code working. Needs clean up, and error message cases like "ownership code not found".Time: ~4 hrs each day
Day 6
Got verified sign up working for bloggers. Yes!Time: ~5 hrs
Day 3
Worked on:- As always something as simple as adding a blog url is complicated. What if has already been added? What if someone else claims ownership? So many questions so little time.
- Extending the MetaCode IDE to support complex validation time queries. It already supports record does and does not exist queries.
Time: ~5 hrs
Postscript: Posted a big question on a blog discovery site on eddit.com/r/Blogging/ No response so far.Day 2
- Added verification code generation to the blog url submission.
- Generating the code, but not yet arriving in the database.
What went well:
Since I did not have a "verification code generator" servlet template I hacked an "update" servlet template to add code generation to it. What is really nice is how well new templates integrate with my MetaCode IDE. The process was:- Cloned the "Update" template java source and xml files
- Added fields like code length, code prefix, code postfix to the xml file.
- Added a few lines of code to call StringHelp.random(codeLength) and do the pre and post appends
- A couple of clicks and the MetaCode IDE imports the new template and builds a matching form
- I fill in the form fields I mentioned, and it generates the servlet code.
What did not go well:
Firefox cost me several hours of wasted time! It just would not refresh with my latest .css file so I wasted the time before I figured that out. What really confused me was, I could see my latest .css code in the Firefox inspector, but that was not what it was rendering with. Chrome worked fine. Lesson learned.Time: ~6 hrs
Day 1
- Modified the "blog" table to include columns for the sitemap and robots files.
- Created a ""Submit blog" form and have it updating the blogs table.
- Need to prefix the "Submit Blog" by providing an embedded code for the blogger to add to the home page. (Need them to prove ownershp)
- Coded a new method for my StringHelp class: static public String random(int length)
Time: 2.5 hrs
Next Steps:- Add a button for the blogger to generate the code.
- Verify the code was added (scrape the web page), check the code update the blog record.
- Grab the sitemap and add it to the blog record.
- Grab the robots.txt and add it to the blog record.
Day 0 Status
Challenge on! I am going to try the 100 Days of Code challenge in building a blog discovery site. We are not talking about a "learn to code" diary here. We are talking a killer web app as powerful as the big platforms like Facebook/Instagram/Pintrest/Twitter, but focused on the blog community. It will be a content (blog) discovery and sharing platform having a mix of search, browse and some social functions.
This is a “from scratch” development. So far I have obtained several potential domain names and activated this one. I have written a couple of test modules for extracting data from blogs. I have put up a “holding” page as the home page and created/uploaded this page.