3.4 some serious scraping.

4h I finished a hackerrank problem that I had been banging my head on tonight, but the bulk of my productivity has been poured into the craigslist scraper. I did some planning and I started building it out in a way that will be useful once I separate everything into a backend and a frontend. Currently I’m hodgepodging it together in express and it’s fairly ugly with lots of html strings in js. I think I have designed it so that I will be able to pull out a lot of it and dump it into a react component in the future. I’m still not certain how the api to front end handoff will go, but I am currently leaning towards a json file that can get rebuilt into a bunch of objects on the front end. It is an interesting mental decision figuring out where the logic should be. For instance: do I want to pass my list of blocked posts to the backend so it can return a clean array, or do I want to get the dirty data from the api and filter it on the frontend? Is data transfer more important, or is frontend processing power? I certainly don’t have the answers yet, but I like figuring out the questions!

I did spend a bit of time today styling the page, and I’m happy with my results so far. I’m getting more comfortable with flexbox slowly and I was able to use it today to do some layout for all of the posts. I think the next level for me is to start to use media queries and then have them blend in together for resizes. Oh yeah, accessibility too. I don’t know much, and I’m sure I am not doing a lot that I could be in order to make my sites easier to use for everyone.

The hackerrank problem was a bit frustrating because I over optimized and overcomplicated things before they were properly working. I ended up ripping it up and doing it with two forEach loops and two counter variables. I was also searching for a substring within each element when I should have just been checking for equality between the two elements. That is also frustrating that I didn’t pick up on it earlier. I started another one right afterwards and I seem to be stuck on an optimization check. My code works on some tests, but not all, and the ones it doesn’t work on time out. It feels to me like I need to find a more efficient way to do everything I am doing. I asked for help on twitter, so we will see if anyone responds.

Tell me what you think.

This site uses Akismet to reduce spam. Learn how your comment data is processed.