All your files are belong to us. [Bringing Remote Files locally]

BoxF · Dec 4, 2017

Hello everyone, BoxF here.

When people often talk about SEO and sitespeed, you rarely hear people talking about remote files aside from... HOLY !#$@%#^ Remove some scripts you crazy monkey!

I've been playing around with some simple proof of concepts which allow you to reduce some of those external requests and still enable you to leverage their usability.

So far here's what I've gotten:

PHP:

<?php
//Place me in your public_html root
$localpath = dirname(__FILE__);
//General Housekeeping
if (!file_exists($localpath.'/js')) {
    mkdir($localpath.'/js', 0777, true);
}
//set local path
$localfile = $localpath.'/js/ga-local.js';
// local analytics path is now /js/ga-local.js don’t forget to update all references of Analytics.js to this path!

$remotefile = 'https://www.google-analytics.com/analytics.js';
$googlemd5 = md5_file($remotefile);
if(file_exists($localfile)){
$localmd5 = md5_file($localfile);
}
if($googlemd5 == $localmd5) {
    echo "You're Local version of google analytics is up to the latest version.)";
    die;
} elseif(copy($remotefile,$localfile)) {
        echo "Success: Ga-local.js File is located here: ".$localfile;
    } else {
        echo "Import of Analytics.js failed.";
    }
?>

But wait..

While this is still a working concept, there are some area's I want to add to this script in order to build it even further.

CRON support for scheduling the updater
Add Byte validation encase remote file moves or get's renamed to prevent NULL data in our local file
Some sort of notification for when imports fail or files move
Array support for supporting more than 1 file at a time
Add Global PHP variables for Script Sources

If any of you guys have experience in similar tests like this I'd love to hear more!
Do you have any advice into roadblocks?

IF NOT...
I'll keep posting my iterations so everyone can leverage bringing all of their scripts locally!

Let's kick some ass!

darkzerothree · Dec 4, 2017

I see what you are trying to do... but this

PHP:

$remotefile = 'https://www.google-analytics.com/analytics.js';
$googlemd5 = md5_file($remotefile);

Still downloads the external file in order to generate the MD5 string.

https://stackoverflow.com/questions...load-the-file-it-is-getting-the-checksum-from

BoxF · Dec 4, 2017

I should have added more context! You're right, but this script is NOT to be included into your index.php or any other page, the only time it gets requested is by the Cron job or manually through a browser or command line, which won't cause any latency. :-)

Thanks for the reply!

Samwise89 · Dec 4, 2017

I'm worried less about Google files than I am files from ad networks. Google is fast and reliable, never had them blocking rendering from fonts, analytics, or jquery.

What all kind of files would this work on? I can't see it working for ad networks but that'd be amazing.

How frequently would you run the cron job? It would suck to have a 30 minute gap where analytics breaks due to some update that you didn't fetch yet.

BoxF · Dec 4, 2017

This will be able to work on any script or resource, that includes fonts (be careful of copyright!), css, js, and even images.

I haven't seen any issues with Analytics, but I can definitely having Analytics misfire would be an extreme piss-off!

For Javascript files (for the most part) you won't have any issues storing a local version as it has to be able to be run from outside of the server its hosted on (as it's executed by the browser) so you shouldn't run into any issues. I would suggest doing the Cron job to run once every 6 hours.

Alongside that note: This is still a Proof-of-concept

So bare with me and let's explore it's limitations :-)

darkzerothree · Dec 4, 2017

aha ...

nice. Altho I think running it every 6 hours is overkill.

BoxF · Dec 4, 2017

darkzerothree said:
aha ...

nice. Altho I think running it every 6 hours is overkill.

It definitely can be! I guess it depends on the type of files you're looking to import :-) I'm not sure exactly what a good schedule is as I have only tested with Analytics. Will definitely keep you guys in the loop of my findings.

turbin3 · Dec 4, 2017

I'd recommend not setting permissions at 0777.

Over the past few years, I was engaged in one massive test, cutting page speed to the absolute bone. I'm talking large-scale site with sets of pages that saw as low as just under 80ms load times, in production...

We got pretty creative with it, and bested Every. Single. Competitor. In the entire niche. Some of them by several hundred percent. It was a fun experiment, to be sure.

The thing I found was, overall speed was less important than stability of speed. Basically, as long as you were under some arbitrary threshold for the niche/competition, things were "good enough". Good enough as in you probably earned all the on-page points G was gonna give you for that one rank factor. Stability being important to not affecting crawl budget, considering these were large sites.

Got to the point where I was generating over 1M static pages in production, if that's any indicator...

I say all of that to preface this next quote.

Premature optimization is the root of all evil.
- Donald Knuth

I don't quote that to be a dick, but rather to potentially save people from a life wasted, searching for another hit off the page speed crack pipe. Trust me on this. I'm a recovering page speed addict. I still like making things fast, light, and providing good UX in the process. That said, I certainly ain't getting any younger. :wink:

Things are certainly different in select niches, such as large ecommerce niches where milliseconds are measured in dollars. In mine, despite the size and competition of the niche, it just wasn't that big of a deal.

So, I hate to come off as negative. I especially like the creativity in thinking you put into this @BoxF, so don't take any of this the wrong way! I just think, beyond some fundamental speed improvements, it's going to be a higher ROI effort focusing on optimizing the ads themselves.

Got a network that's insane with HTTP requests and resource overhead? Find better networks. Even better, cut out the middleman and find brand/mfr direct affiliate programs if they're available. ROI multiplied!

Considering the nature of some of these networks, with their 200+ HTTP requests on a single page load, I honestly think they're gonna be forced to evolve or go the way of broadcast television. It's almost 2018. Ain't nobody got time 'fo that! LOL

BoxF · Dec 5, 2017

just under 80ms load times, in production...

That's truly incredible, any tips you can share in some of those fun tests you did to achieve this? I'm sure it'll help some of the people following this, as well inspire some more tests for me.

I especially like the creativity in thinking you put into this @BoxF, so don't take any of this the wrong way! I just think, beyond some fundamental speed improvements, it's going to be a higher ROI effort focusing on optimizing the ads themselves.

Appreciate it! Definitely going to pick your brain a bit more about the ad side.

I'm more of a curious SEO rather than a die-hard affiliate advertiser, strength in one and opportunity to learn in the other. I would love to hear more about the tests you guys were able to achieve on the larger end, I have a few test sites with similar amounts of indexed pages that I like to throw these on.

Considering the nature of some of these networks, with their 200+ HTTP requests on a single page load, I honestly think they're gonna be forced to evolve or go the way of broadcast television. It's almost 2018. Ain't nobody got time 'fo that! LOL

LOL! I agree with you, we've witnessed quite a bit of change (in Google, in the Internet, in the mind) over the years. It would surely be naive to assume that the current working version of the Internet will exist in the near future, after-all it was never supposed to be mass-marketed.

@turbin3 - I hope you don't mind, I'll reach out to you privately to start a discussion, really love where your mind is at.

I really appreciate the comments on this thread so far. You all are truly my kin. Much respect! :evil:

*disappears back into the darkness*

turbin3 · Dec 5, 2017

BoxF said:
That's truly incredible, any tips you can share in some of those fun tests you did to achieve this? I'm sure it'll help some of the people following this, as well inspire some more tests for me.

Some of the things I did were along similar lines to your original point. Namely self-hosting resource files.

I've tried it before with Google Analytics, but honestly was a bit too concerned about data integrity to keep it self-hosted.

The core factors I usually work on are:

Reducing HTTP requests
Reducing the actual HTML file size if possible
Async as much JS as possible, or strip it out if not absolutely needed
- One example is some themes, frameworks, etc. just load "all" the resource files on all pages. Some pages may not actually need them.
- With WP and child themes, I might make a new page template specifically for that purpose (in the child theme), and strip it to the bone.
Lazy load images. It improves render time, can reduce initial number of requests, and generally improves UX overall.
- A LOT of methods for this. Most usually load what's above the fold and only load below the fold images when scrolled in view.
- Some also use placeholder images, low res image versions, base64 data URIs, among other things.

The thing I've been enjoying the most lately, and what I HIGHLY recommend people play around with, is the new CSS grid specification. It's downright amazing. It will be the death blow to the #divception of typical CSS frameworks. This accomplishes several things.

Makes code and page structure easy to reason about (extremely important productivity factor)
Reduces overall code size
Makes it easy to just use semantic HTML5 elements and make some really CLEAN code
You can totally change layout on the fly, faster than any framework. PERIOD.

Example. Let's look at the code for the "At Your Service" content block for this Start Bootstrap template:

Code:

<section id="services">
  <div class="container">
    <div class="row">
      <div class="col-lg-12 text-center">
        <h2 class="section-heading">At Your Service</h2>
        <hr class="my-4">
      </div>
    </div>
  </div>
  <div class="container">
    <div class="row">
      <div class="col-lg-3 col-md-6 text-center">
        <div class="service-box mt-5 mx-auto">
          <i class="fa fa-4x fa-diamond text-primary mb-3 sr-icons"></i>
          <h3 class="mb-3">Sturdy Templates</h3>
          <p class="text-muted mb-0">Our templates are updated regularly so they don't break.</p>
        </div>
      </div>
      <div class="col-lg-3 col-md-6 text-center">
        <div class="service-box mt-5 mx-auto">
          <i class="fa fa-4x fa-paper-plane text-primary mb-3 sr-icons"></i>
          <h3 class="mb-3">Ready to Ship</h3>
          <p class="text-muted mb-0">You can use this theme as is, or you can make changes!</p>
        </div>
      </div>
      <div class="col-lg-3 col-md-6 text-center">
        <div class="service-box mt-5 mx-auto">
          <i class="fa fa-4x fa-newspaper-o text-primary mb-3 sr-icons"></i>
          <h3 class="mb-3">Up to Date</h3>
          <p class="text-muted mb-0">We update dependencies to keep things fresh.</p>
        </div>
      </div>
      <div class="col-lg-3 col-md-6 text-center">
        <div class="service-box mt-5 mx-auto">
          <i class="fa fa-4x fa-heart text-primary mb-3 sr-icons"></i>
          <h3 class="mb-3">Made with Love</h3>
          <p class="text-muted mb-0">You have to make your websites with love these days!</p>
        </div>
      </div>
    </div>
  </div>
</section>

Not terrible. Kind of a lot of boilerplate still surrounding the actual meat and potatoes content though.

Ready to have your mind blown? Check out a CSS grid version that could do the same thing:

Code:

<section id="services">
  <h2>At Your Service</h2>
  <div class="service-box">
    <h3>Sturdy Templates</h3>
    <p>Our templates are updated regularly so they don't break.</p>
  </div>
  <div class="service-box">
    <h3>Ready to Ship</h3>
    <p>You can use this theme as is, or you can make changes!</p>
  </div>
  <div class="service-box">
    <h3>Up to Date</h3>
    <p>We update dependencies to keep things fresh.</p>
  </div>
  <div class="service-box">
    <h3>Made with Love</h3>
    <p>You have to make your websites with love these days!</p>
  </div>
</section>

From 42 lines down to 19. 55% reduction!

And here's the basic CSS, excluding media queries and icons. For the icons, those could be done using :before and "content" in CSS, as well as nth-child or probably a few other methods.

Code:

  #services {
    display: grid;
    grid-template-columns: repeat(4, 1fr);
    grid-template-rows: auto;
    grid-gap: 2em;
    text-align: center;
  }

  #services h2 {
    grid-column: span 4;
  }

You could even chop a few lines off CSS using the "grid" shorthand to handle rows and columns in one line:

Code:

  #services {
    display: grid;
    grid: auto / repeat(4, 1fr);
    grid-gap: 2em;
    text-align: center;
  }

  #services h2 {
    grid-column: span 4;
  }

That's about as simple as I'd go. You could probably strip another few lines out, but the CSS will get a bit more complex. See, the idea with CSS grid is, handle design and layout in CSS so you can reduce things being heavily customized in HTML. Even better, with just a line or 2 of changes in CSS, you can move those boxes around entirely and create a totally different layout or design!

As far as media queries, it gets handled the opposite way most do them. Mobile first. So you start with a minimum browser size and everything under it just stacks usually. That's the nice thing about clean, semantic, HTML5 is it's really easy to code your structure to gracefully degrade and, when in doubt, the page components just stack like mobile!

Resources:
CSS grid Bible
Rachel Andrew Grid By Example

All your files are belong to us. [Bringing Remote Files locally]

BoxF

ǝʌıʇɔǝdsɹǝd ɹnoʎ ǝɓuɐɥɔ

darkzerothree

DunkelNullDrei

BoxF

ǝʌıʇɔǝdsɹǝd ɹnoʎ ǝɓuɐɥɔ

Samwise89

BoxF

ǝʌıʇɔǝdsɹǝd ɹnoʎ ǝɓuɐɥɔ

darkzerothree

DunkelNullDrei

BoxF

ǝʌıʇɔǝdsɹǝd ɹnoʎ ǝɓuɐɥɔ

turbin3

BoxF

ǝʌıʇɔǝdsɹǝd ɹnoʎ ǝɓuɐɥɔ

turbin3