How do Giant Web Sites Using Static Site Generators Update Frequently Throughout the Day?

[Raymond Luo]

How do SSG devs or publishers like The Verge (who uses Middleman) routinely update their site over the course of the day?

Do they cleanse and upload en-masse or in small batches each time?

 

[GrandMaster]

SSG

You should probably define the non-standard acronym since you are in the newbie section. Jargon helps no one. I don't have the slightest clue what SSG stands for even after Googling it.

 

[mbzzt]

SSG = Static Site Generators?

 

[Steve Brownlie]

How do SSG devs or publishers like The Verge (who uses Middleman) routinely update their site over the course of the day?

SSG = Static Site Generators?

Yes.

Just a heads up for non SEOs/search people among the newbies. The easy way with things like this is to just mix the key points from the sentence into a search query and Google will naturally give you the answer. I didn't know what Middleman was but:

"middleman SSG"

Yielded no1 result:

Search Results
Middleman: Hand-crafted frontend development
https://middlemanapp.com/
Build static websites with an easy-to-use framework. Middleman is a static site generator using all the shortcuts and tools in modern web development.

 

[Raymond Luo]

Sorry about that earlier. And thanks @Steve Brownlie for clearing that up.

Obligatory Explanation

I should explain myself a little. WordPress is a smart and dynamic. Every time a user enters a WP website, it compiles and processes what needs to be shown to that user.

You can imagine if a ton of users needs individual compilation and processing of the same page, it would be a lot of work.

That's where you've probably heard of caching too, which saves the result of the compilation and processing of the page and sends it to the user. That's very static-like.

Static File Generators are similar except they are not 'like' static, they are. There are many benefits to this and the cons relate more towards workflow. You can read more online.

WordPress are a dream and very easy to use until they don't anymore because of $insert reasons. They become slow, vulnerable or inextensible. If you experience any of those problems and actually want to solve them, you might want to consider maintaining your WP site at an obscure location and generate a static site on your WP.

I happened to chance on some solutions like http://www.simplystatic.co/. I am not recommending them and have not used them.

Almost Back To My Question

I have never used an SSG and probably not in the near future because the workflow seems highly inefficient. Look at the amount of work at this really good article: https://www.smashingmagazine.com/2016/08/using-a-static-site-generator-at-scale-lessons-learned/.

Back To My Question (Linux Commands)

But my current CMS (Grav) has plugins that can generate static sites, which is why I chose it. The plugin can generate specific directories of the site, or it can generate everything. So at some point, I need to transfer the generated directories into the /public directory.

It looks like there are cp and rsync commands available that I can use to transfer on the cron. But in a production environment, I'm not sure which of these (or others) will be efficient considering at scale.

I know this deviates from the original question, mainly because I kind of answered my own question in writing this and I'm left with a technical Linux based question.

 

[turbin3]

With SSG's, what works and what will be efficient really depends on the degree of functionality that the particular site actually needs. In a lot of cases, I think people overestimate the need for a highly dynamic, database-driven site. For example, in a lot of cases, people might think the technology they need is dynamic/database-oriented, when the reality may be they simply need more functionality and more progressive UX with some sort of JS framework to further customize the user's experience.

I think the real trick is determining, for the individual use case, where your threshold is between data-driven content and design. What I mean by that is, if you need to iterate over and generate things from datasets, some SSGs can still be efficient with this. The trick is whether you only need a moderate amount or a high degree of this. For example, with some SSGs, you can effectively turn certain file types like YAML, TOML, or JSON (of course) effectively into large arrays of data. Posts, pages, or components can be generated from this. For low to moderate levels of data (thinks tens or thousands of posts, pages, etc.), with certain SSGs, this is not an issue and can be done efficiently. If we're talking hundreds of thousands, or millions, in certain very specific use cases, SSGs may still be feasible, but that's database/dynamic level at that point, if you want to make things easy. In one VERY specific use case for a very specific purpose, I have a static site in production that is regenerating and serving over 1M static pages weekly...

Workflow is the other major issue of course, as you mentioned. By far, Hugo is my favorite when it comes to SSG's. Combine it with NPM, Gulp, and maybe a few other plugins, then create a few script commands of your own in your package.json, and it's pretty easy and efficient to streamline your dev/build/deploy process. If you need a front end for content writers, there are more options for that these days, though that still remains one of the major issues that continues to hold the SSG world back from becoming more mainstream.