Tag Archive | "Let’s"

Target Lets You Pay With New ‘Wallet’ Mobile Payment System

It appears that Target is still not keen on accepting Apple Pay as a mode of payment in its onsite stores. In a recent post, America’s second-largest retailer announced that it had rolled out Wallet, its own mobile payment system that is now active as a feature for Target’s iOS and Android apps.

The arrival of the Wallet feature has actually been long anticipated by customers. Talks about Target launching its own mobile payment have been around since 2015, but the company waited till January 2017 to confirm its plans.

With the introduction of Wallet, Target customers will now have the option of seamlessly checking out their in-store purchases with the use of their smartphones. They’ll also be able to update points and use discounts with Cartwheel digital coupons.

Reportedly, the Wallet feature found in the Target app is similar to other in-store payment options like Wal-Mart Pay. However, Wal-Mart Pay uses QR code for scanning items, Target’s system is barcode-based.

For it to work though, it seems that shoppers will need a REDcard—a Target-branded debit or credit card packed with special offers. Target, however, wants more customers to shop with Wallet and is already working out a way to get them to use the feature even if they don’t own a REDcard.

The main incentive for using Wallet is that checking out for in-store purchases would be faster. This is achieved because customers can now check out for payment as well as apply Cartwheel deals with a single scan of the in-app barcode.

It should be noted that while Target does not accept payments using Apple Pay for in-store purchases, items bought from Target’s online store can still be paid using Apple’s payment platform.

[Featured Image by Target]

The post Target Lets You Pay With New 'Wallet' Mobile Payment System appeared first on WebProNews.


WebProNews

Posted in IM NewsComments Off

Google Search Has a New Feature That Lets You Compare Devices

Google is testing out a new feature to its popular search engine that should make gadget or device-related queries even more interesting. Apparently, the tech giant is planning to roll out a device comparison feature that lets you compare the specs of two gadgets at a glance.

Of course, Google has been known to introduce changes to the search engine in the past without prior announcement. It seems that the company introduced the device comparison feature in a similar manner but some observant users were able to note the changes nevertheless.

According to Android Police, the device comparison feature is activated by placing “vs” in the middle of the two devices being compared. So, if you want to compare the specs between Google’s latest smartphone offerings, just type “Pixel 2 vs Pixel 2 XL.” However, the feature also lets you compare specs of devices with different brands, which means you can also type an iPhone model and see how it stacks up against its Samsung rival.

After you hit the send button, the search result will show the specs of the two devices in a condensed format, GoAndroid reported. However, the view can be expanded by tapping a blue button to show the full comparison. You can also toggle between the views by pressing the same button.

The new feature will compare user reviews, price range, available colors of the gadget, the date the device was released as well as its screen size. For performance conscious consumers, the search result will likewise reveal battery capacity, weight, pixel density, width, battery life, operating system, storage capacity, camera resolution, supported cellular network, brand as well as connectivity features.

Right now, the device comparison feature is only available for the mobile version of the search engine. In addition, Google might still be in the process of testing the new feature as some mobile users have reported that they were unable to make side-by-side comparisons on their particular devices.

[Featured Image by Pixabay]

The post Google Search Has a New Feature That Lets You Compare Devices appeared first on WebProNews.


WebProNews

Posted in IM NewsComments Off

Whatsapp Introduces New Live Location Feature That Lets You Track Friends in Real Time

Pretty soon, it will be easier for family members and close friends to track the location of their loved ones to make sure they have safely reached their destination. WhatsApp has introduced a new feature that gives its users the ability to broadcast their location in real time.

Called Live Location, the new WhatsApp feature will be available on both Android and iOS versions of the application. Apparently, it is a feature that is similar to that of parent firm Facebook, which rolled out the temporary location sharing feature on Messenger last May.

To share your location temporarily with someone, you need to open a chat with that friend or family member on your contact list. The feature can be accessed by choosing the Location option within chat, and you will then need to choose the length of time you want your location to be known to the other party. The feature will start airing your location in real time once you hit the send button.

At the moment, there seem to be three options for the duration of the Live Location feature, according to Tech Crunch. Users can share their real-time location for fifteen minutes, one hour or eight hours but they can manually turn off the feature if they no longer want their location to be broadcasted or if they have already reached their destination safely.

Meanwhile, the receiving party will see a map with the broadcaster’s avatar at the center on their chat. There is also a “Live Until” message at the bottom indicating the time that the Live Location tracking will end.

With its introduction of the Live Location, WhatsApp has joined Snapchat and Foursquare which already have their own versions of the feature.

[Featured Image via YouTube]

The post Whatsapp Introduces New Live Location Feature That Lets You Track Friends in Real Time appeared first on WebProNews.


WebProNews

Posted in IM NewsComments Off

Pros and Cons of HTTPS Services: Traditional vs Let’s Encrypt vs Cloudflare

Posted by jrridley

If you have a website property verified in Google Search Console, and the website is not HTTPS-secured, you’ve likely seen some form of the following message in your dashboard recently:

After months of talk and speculation, Google has finally started to move forward with its plan to secure the web by enforcing HTTPS. Although HTTPS had previously only been a concern for e-commerce sites or sites with login functionality, this latest update affects significantly more sites. The vast majority of websites have a contact page (or something similar) that contains a contact or subscription form. Those forms almost always contain text input fields like the ones Google warns about in the message above. The “NOT SECURE” warning has already been appearing on insecure sites that collect payment information or passwords. It looks like this in a user’s URL bar:

Now that this warning will be displaying for a much larger percentage of the web, webmasters can’t put off an HTTPS implementation any longer. Unfortunately, Google’s advice to webmasters for solving this problem is about as vague and unhelpful as you might imagine:

Thanks, Google.

Implementing HTTPS is not a simple process. The Washington Post published a blog post outlining their 10-month HTTPS migration back in 2015, and numerous sites (including Moz) have reported experiencing major traffic fluctuations following their migrations. The time and resources required to migrate to HTTPS are no minor investment; we’re talking about a substantial website overhaul. In spite of these obstacles, Google has shown little sympathy for the plight of webmasters:

Google’s singular focus in this area is to provide a better user experience to web visitors by improving Internet security. On its surface, there’s nothing wrong with this movement. However, Google’s blatant disregard for the complexities this creates for webmasters leaves a less-than-pleasant taste in my mouth, despite their good intentions.

Luckily, there’s a bit of a silver lining to these HTTPS concerns. Over the last few years, we’ve worked with a number of different clients to implement HTTPS on their sites using a variety of different methods. Each experience was unique and presented its own set of challenges and obstacles. In a previous post, I wrote about the steps to take before, during, and after a migration based on our experience. In this post, my focus is instead on highlighting the pros and cons of various HTTPS services, including non-traditional implementations.

Here are the three methods we’ve worked with for our clients:

  1. Traditional HTTPS implementation
  2. Let’s Encrypt
  3. Cloudflare


Method 1: Traditional HTTPS implementation

A traditional HTTPS implementation starts with purchasing an SSL certificate from a trusted provider, like Digicert or Comodo (hint: if a site selling SSL certificates is not HTTPS-secured, don’t buy from them!). (*NOTE: Google just announced this week they will no longer trust certificates issued by Symantec, which includes the brands Thawte, VeriSign, Equifax, GeoTrust, and RapidSSL.) After that, you’ll need to verify the certificate with the Certificate Authority you purchased it from through a Certificate Signing Request (CSR); this just proves that you do manage the site you claim to be managing. At this point, your SSL certificate will be validated, but you’ll still have to implement it across your site. Namecheap has a great article about installing SSL certificates depending on your server type. Once that SSL certificate has been installed, your site will be secured, and you can take additional steps to enable HSTS or forced HTTPS rewrites at this point.

Pros

  1. Complete security. With a fully validated SSL certificate installed on your root server, there is no possibility of having a compromised connection between your server and site, or between your site and the site visitor.
  2. Customizable. One of the features of a full SSL implementation is that you can purchase an Extended Validation (EV) SSL certificate. This not only provides your green padlock in the browser bar, but also includes your company name to provide further assurance to visitors that your site is safe and secure.
  3. Easier to implement across multiple subdomains. If you have multiple subdomains, what you’ll likely need for your HTTPS implementation is either a separate SSL certificate for each subdomain or a wildcard certificate for all variations of your domain. A traditional SSL service is often the easiest way to set up a wildcard certificate if you need to secure several variations.

Cons

  1. Expensive. Though basic SSL certificates may be available for as little as $ 150, depending on the complexity of your site, these costs can quickly increase to several thousand dollars if you need more advanced security features, a better CDN network, etc. This also doesn’t include the cost of having developers implement the SSL certificate, which can be extensive as well.
  2. Time to implement. As mentioned above, it took the Washington Post 10 months to complete their HTTPS migration. Other companies have reported similar timeframes, especially for larger, more complex websites. It’s very hard to know in advance what kinds of issues you’ll have to resolve with your site configuration, what kinds of mixed content you may run into, etc., so plan lots of extra time to address these issues if you go with a standard implementation.


Method 2: Let’s Encrypt

Let’s Encrypt is a free nonprofit service provided by the Internet Security Research Group to promote web security by providing free SSL certificates. Implementing Let’s Encrypt is very similar to a traditional HTTPS implementation: You still need to validate the Certificate Authority, install the SSL certificate on your server, then enable HSTS or Forced HTTPS rewrites. However, implementing Let’s Encrypt is often much simpler through the help of services like Certbot, which will provide the implementation code needed for your particular software and server configuration.

Pros

  1. Free. The cost is zero, zippo, nada. No fine print or hidden details.
  2. Ease of implementation. Let’s Encrypt SSL is often much simpler to implement on your site than a traditional HTTPS implementation. Although not quite as simple as Cloudflare (see below), this ease of implementation can solve a lot of technical hurdles for people looking to install an SSL certificate.
  3. Complete security. Like with a traditional HTTPS implementation, the entire connection between site visitor and site server is secure, leaving no possibility of a compromised connection.

Cons

  1. Compatibility issues. Let’s Encrypt is known to be incompatible with a few different platforms, though the ones it is incompatible with are not likely to be a major source of traffic to your site (Blackberry, Nintendo 3DS, etc.).
  2. 90-day certificates. While traditional SSL certificates are often valid for a year or more, Let’s Encrypt certificates are only valid for 90 days, and they recommend renewing every 60 days. Forgetting to renew your certificate with this necessary frequency could put your site in a compromising situation.
  3. Limited customization. Let’s Encrypt will only offer Domain Validation certificates, meaning that you can’t purchase a certificate to get that EV green bar SSL certificate. Also, Let’s Encrypt does not currently offer wildcard certificates to secure all of your subdomains, though they’ve announced this will be rolling out in January 2018.

Method 3: Cloudflare

This is one of my favorite HTTPS implementations, simply because of how easy it is to enable. Cloudflare offers a Flexible SSL service, which removes almost all of the hassle of implementing an SSL certificate directly on your site. Instead, Cloudflare will host a cached version of your site on their servers and secure the connection to the site visitors through their own SSL protection. You can see what this looks like in the picture below:

In doing so, Cloudflare makes this process about as simple as you can ask for. All you have to do is update your DNS records to point to Cloudflare’s nameservers. Boom, done. And as with Let’s Encrypt, the process is entirely free.

Pros

  1. Free. The cost is zero, zippo, nada. No fine print or hidden details. Cloudflare does offer more advanced features if you upgrade to one of their paid plans, but the base SSL service comes completely free.
  2. Easiest implementation. As I mentioned above, all that’s required for implementing Cloudflare’s SSL service is creating an account and updating your DNS records. There’s no update to the server configuration and no time spent trying to resolve additional configuration issues. Additionally, implementing HSTS and forced HTTPS rewrites can be done directly through the Cloudflare dashboard, so there’s really almost no work involved on your end.
  3. PageSpeed optimizations. In addition to SSL security, Cloudflare’s HTTPS implementation also provides several additional services that can preserve PageSpeed scores and page load times. While a traditional HTTPS implementation (or Let’s Encrypt) can often have negative consequences for your site’s page load times, Cloudflare offers the ability to auto-minify JS, CSS, and HTML; Accelerated Mobile Pages (AMP); and a Rocket loader for faster JS load times. All of these features (along with Cloudflare serving a cached version of your site to visitors) will help prevent any increase in page load times on your site.

Cons

  1. Incomplete encryption. As you can see in the picture above, Cloudflare encrypts the connection between the visitor and the cached version of your site on Cloudflare, but it doesn’t encrypt the connection between your site and your server. While this means that site visitors can feel secure while visiting your site, there is still the chance that your server connection will be compromised. While you can upgrade to a full SSL implementation that does enable this setup, that is not part of the free service.
  2. Security concerns. Cloudflare was infamously hacked earlier this year, exposing lots of sensitive user information. While it appears they have resolved and tightened security since then, it’s still important to be aware of this development.
  3. Lack of customization. Like with Let’s Encrypt, Cloudflare’s free SSL service doesn’t provide any kind of EV green bar SSL for your site. While you can upgrade to full SSL which does provide this functionality, the service is no longer free at that point.

Which type of HTTPS implementation is best?

It really depends on your site. Smaller sites who just need enough security that Google won’t punish the site in Chrome can likely use Cloudflare. The same goes for agencies providing HTTPS recommendations to clients where you don’t have development control of the site. On the other hand, major e-commerce or publication sites are going to want a fully customized HTTPS implementation through traditional means (or via Let’s Encrypt’s wildcard certificate, when that happens next year). Ultimately, you’ll have to decide which implementation makes the most sense for your situation.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!


Moz Blog

Posted in IM NewsComments Off

Bing’s “My Saves” feature lets users save video, image and shopping searches across devices

The new feature lets users save an image, video or shopping search result to their “My Saves” folder which can be accessed across mobile devices and desktop.

The post Bing’s “My Saves” feature lets users save video, image and shopping searches across devices appeared first on…



Please visit Search Engine Land for the full article.


Search Engine Land: News & Info About SEO, PPC, SEM, Search Engines & Search Marketing

Posted in IM NewsComments Off

The Two-Part SEO Ranking Model: Let’s Make SEO Simple

Posted by EricEnge

There sure is a lot of interest in SEO ranking factors:

There have been major studies done on this, notably by both Moz and Searchmetrics. These are groundbreaking pieces of research, and if you’re serious about SEO, you need to understand what these studies say.

That said, these are too complex for most organizations to deal with. They need a simpler way of looking at things. At Stone Temple Consulting (STC) we deal with many different types of organizations, including some of the world’s largest companies, and some of the highest-traffic websites in the world. For most of these companies, understanding that there are 200+ ranking factors does more harm than good.

Why, you ask? So many people I talk to are looking for a silver bullet. They want to hear that they should only change their heading tags on the second Tuesday of every month, except during leap years, when they should do it on the first Tuesday, except in February when they should change it on the third Monday. These distractions end up taking away from the focus on the two things that matter most: building great content (and a great content experience) and promoting it well.

Today’s post is going to lay out a basic approach that most companies can use to simplify their thinking about SEO, and keep their focus on the highest priorities.

What Google recently said

Here’s what Google Dublin’s Andrey Lippatsev said in a Hangout that I participated in on March 23, 2016. Also participating in the Hangout was Ammon Johns, who asked Andrey what the two most important ranking factors are:

Andrey Lippatsev: Yes. Absolutely. I can tell you what they are. It is content and links going into your site.

There we go, that’s a start. According to Google, it’s links and content that are the two biggest. Hopefully, the idea that content is a big factor is obvious, but below I’ll break out more what great content really entails. In addition, you can see some backup for the power of links in the study I recently published on links as a ranking factor.

Should we think of the world as consisting only of these two factors? It’s quite simplistic, and possibly too much so, but let’s try to simplify this even more. How many organizations would dramatically improve their SEO if they focused on creating great content and promoting it effectively? I can tell you that from my experience these are two things that many organizations simply don’t do.

Does that mean that we can take our two factors and put them into a (purely) hypothetical ranking score equation that looks like this?

I actually think that this equation is pretty effective, though it has some limitations and omissions that I’ll describe in more detail below. You also need to think about the concept of “great content,” that will get a high Content Score, in the correct manner.

What is “great content?”

If we step back and think about what makes up great content, it seems to me that there are three major components that matter:

  1. Relevancy
  2. Quality
  3. The overall content experience

The first part of this is simple. If the content is not relevant to a query, it shouldn’t rank for that query, ever. That makes sense, right?

The second part is also pretty simple, and that’s the notion of quality. Does it provide information that people are looking for? Is that information relatively unique to your site? Clearly, it makes sense for the quality of the content to matter a lot.

We can combine the notions of quality and relative uniqueness into the notion of material differentiation. Rand covers this brilliantly in his Whiteboard Friday about creating 10X content.

Creating the 220,001st article on how to make French toast is just not going to cut it:

You need to create something new and compelling that also offers a lot of value. That may not be easy, but being the best at something never is.

If you’re in a competitive market, it’s reasonable to guess that your top competitors are making great, relevant content on topics that matter to their target audience. For the most important queries, it’s probable that the top 5 (and maybe more) pieces of content in that space are really, really good (i.e. more comprehensive than other articles on the topic, or brings in new information that others don’t have).

The third part encompasses many pieces.

  • Is your content well-organized and easy to read?
  • Does it effectively communicate its key points? How do people engage with it? If they land on a page on your site that has the answer to their question, can they quickly and easily find that information?

Once again, you’ll find that the major competitors that rank in the top of the SERPs all handle this pretty well too.

Let’s now take a look at what the role of the content score in ranking might look like:

Note that the Y-axis is “Chances of Ranking,” as opposed to “Ranking.” Nonetheless, this curve suggests that the Content Score is a big one, and that makes sense. Only the best of the best stuff should rank. It’s simple.

Digging a bit deeper on what goes into content quality

But what about title tags? Heading tags, use of synonyms? Page layout and design? Stop and think about it for a moment. Aren’t those all either part of creating higher-quality content, or making that content easier to consume?

You bet.

For example, imagine that I wrote this piece of content:

It could be the greatest information in the world, but it’s going to be really hard for users to read, and it will probably have terrible user engagement signals. On the other hand, imagine that my content looks like this:

Would you say the quality of one of these pieces of content is higher? I would. The second one is much easier to read, and therefore will deliver more value to users. It will get better engagement, and yes, it will probably get linked to more often.

Why do links get separate treatment?

You could argue that links are just another measurement of content quality, and there is some truth to that, but we give them separate treatment in this discussion for two reasons:

1. They’re still the best measurement of authority.

Yes, I know I’m ruffling some feathers now, but this is what my experience after more than 15 years in SEO (and seeing hundreds of SEO campaigns) has taught me. To get and sustain a link, someone has to have a website, has to be willing to modify that website, and they have to be willing to have their site’s visitors click on the link to leave their site and go to yours.

That’s a pretty material commitment on the linking site’s part, and the only incentive they have to do that is if they believe that your content is of value to their site’s visitors.

Why not social signals? While I’ve long argued that they have no impact except for aiding in content discovery, let’s for sake of argument say that I’m wrong, and there is some impact here, and explain why social signals can never be a critical part of the Google algo. It’s simple: social signals are under the control of third-party companies that can make them invisible to Google on a moment’s notice (and remember that Google and Facebook are NOT friends). Imagine Google giving Facebook (or any other 3rd party) the power to break their algorithm whenever they want. Not happening!

2. The power of links should cause different actions on your part.

What is that action? It’s called marketing, and within that discipline is the concept of content marketing. Done the right way, these are things you should do to raise the reputation and visibility of your brand.

In fact, this may consume a material amount of your entire company budget. With or without search engines in the world, you’ve always wanted to do two things:

(1) Make really good stuff, and

(2) market it effectively.

In 2016, and beyond, this will not change.

No doubt, part of attracting great links is to produce great content, but there are other overt actions involved to tell the world about that great content, such as active outreach programs.

Expanding on user engagement

Many have speculated that Google is using user engagement signals as a ranking factor, and that it will increase its investment in these areas over time. For example, what about click-through rate (CTR)? I discuss CTR as a ranking factor here, but to net it out simply, it’s just too easy a signal to game, and Google tells us that it uses CTR measurements as a quality control check on other ranking signals, rather than as a direct signal.

You can doubt Google’s statements about this, but if you own or publish a website, you probably get many emails a week offering to sell you links via one scheme or another. However, you never get emails offering you CTR ranking schemes. Why is that, you think? It’s because even the scammers and spammers don’t think it works.

Important note: Rand has done many live CTR tests and a number of these have shown some short-term rankings movement, so CTR could be used in some manner to discover hot trends/news, but still not be a core ranking factor.

What about other user engagement signals? I’d bet that Google is, in fact, doing some things with user engagement signals, though it’s hard to be sure what they are. It’s not likely to be as simple as bounce rate, or its cousin, pogosticking.

Pogosticking sure seems like a good signal until you realize there are many scenarios where they don’t work at all. For example, when users are comparison shopping, they’ll naturally hop from site to site.

Finding good user engagement factors that make for really reliable signals is quite hard. Many have speculated that artificial intelligence/machine learning will be used to derive these types of factors. Here are three pieces of content that cover that topic in some detail:

  1. The Machine Learning Revolution: How it Works and its Impact on SEO, an article here on Moz by yours truly
  2. SEO in a Two-Algorithm World, a Powerpoint by Rand Fishkin
  3. The Past, Present, and Future of SEO, an article by Mike Grehan

Information architecture

Having a solid information architecture (IA) that Google can crawl and easily find your content is also a major requirement. In Andrey Lippatsev’s response, he undoubtedly presumed that this was in good shape, but it would be wrong to leave this out of this discussion.

At Stone Temple Consulting, we’ve helped tons of sites improve their organic traffic simply by working on their IA, eliminating excessive page counts, improving their use of SEO tags like rel=canonical, and things of this nature. This is clearly a big factor as well. Usability also feeds into IA, because people need to be able to find what they’re looking for on your site.

What I’ve left out with the two-factor model

First of all, there are other types of results, such as images, videos, and maps results, that are opportunities to get on the first page, but the above discussion is focused on how to rank in regular web search results.

To be fair, even in the regular web results, I’ve left some things out. Here are some examples of those:

  1. Local links. I’m not referring to “local pack” listings here. If I search on “digital cameras” right now, in the regular web search results, I’ll see some listings for stores near me. Clearly, proximity is a very large factor in ranking those pages.
  2. Query deserves diversity. An example of this is the query “Jaguar.” Chances are that my two-factor algorithm would rank only car sites in the top 10, but Google knows that many people that type that query want information on the animal. So even if the two-factor algo would slant things one way, you’ll see some animal-related sites in the top 10.
  3. In-depth articles. This is a feature that’s hard to spot in the search results, but sometimes Google includes in the bottom of the top 10 results some pieces of content that are particularly comprehensive. These are for queries where Google recognizes there’s a decent chance that the user is engaging in extensive research on a topic. Here’s an example for the query “constitution”:

We conducted a small sample review of 200 SERPs and found that about 6% of the results appeared to be from factors such as these. The two-factor model also doesn’t account for personalization, but this post is looking at ranking factors for regular search results other than personalization, which, of course, also has a large impact.

Looking for ranking hacks?

OK, I’m going to give you one. Make your content, and the experience of consuming that content, unbelievably good. That’s step one. Stick to your knitting, folks, and don’t cop out on the effort to make your content stand out. You have no choice if you want to get sustainably positive results from SEO.

Don’t forget the overall site and page usability, as that’s a big part of what makes your content consumable. This is a critical part of making great content. So is measuring user engagement. This provides a critical feedback loop into what you’re doing, and whether or not it’s working for your target audience.

Then, and only then, your focus should turn to marketing that will help drive your reputation and visibility, and help attract links to your content. Here it is in a nutshell:

If your content isn’t competitive in relevance and quality, links won’t help. If it is, links will make the difference.

Your content has to be elite to have a chance to score highly on any given competitive search result. After that, your superior marketing efforts will help you climb to the top of the heap.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!


Moz Blog

Posted in IM NewsComments Off

Google launches iOS keyboard that lets you search in any app & more

Gboard, a new keyword for iOS was launched today by Google to enable searching directly in your iOS keyboard – without leaving the app.

The post Google launches iOS keyboard that lets you search in any app & more appeared first on Search Engine Land.



Please visit Search Engine Land for the full article.


Search Engine Land: News & Info About SEO, PPC, SEM, Search Engines & Search Marketing

Posted in IM NewsComments Off

Google Image Search Lets You Save Images For Later With Stars

Google brings back starring for saving image search results to your photo collections.

The post Google Image Search Lets You Save Images For Later With Stars appeared first on Search Engine Land.



Please visit Search Engine Land for the full article.


Search Engine Land: News & Info About SEO, PPC, SEM, Search Engines & Search Marketing

Posted in IM NewsComments Off

Good News: We Launched a New Index Early! Let’s Also Talk About 2015′s Mozscape Index Woes

Posted by LokiAstari

Good news, everyone: November’s Mozscape index is here! And it’s arrived earlier than expected.

Of late, we’ve faced some big challenges with the Mozscape index — and that’s hurt our customers and our data quality. I’m glad to say that we believe a lot of the troubles are now behind us and that this index, along with those to follow, will provide higher-quality, more relevant, and more useful link data.

Here are some details about this index release:

  • 144,950,855,587 (145 billion) URLs
  • 4,693,987,064 (4 billion) subdomains
  • 198,411,412 (198 million) root domains
  • 882,209,713,340 (882 billion) links
  • Followed vs nofollowed links
    • 3.25% of all links found were nofollowed
    • 63.49% of nofollowed links are internal
    • 36.51% are external
  • Rel canonical: 24.62% of all pages employ the rel=canonical tag
  • The average page has 81 links on it
    • 69 internal links on average
    • 12 external links on average
  • Correlations with Google Rankings:
    • Page Authority: 0.307
    • Domain Authority: 0.176
    • Linking C-Blocks to URL: 0.255

You’ll notice this index is a bit smaller than much of what we’ve released this year. That’s intentional on our part, in order to get fresher, higher-quality stuff and cut out a lot of the junk you may have seen in older indices. DA and PA scores should be more accurate in this index (accurate meaning more representative of how a domain or page will perform in Google based on link equity factors), and that accuracy should continue to climb in the next few indices. We’ll keep a close eye on it and, as always, report the metrics transparently on our index update release page.

What’s been up with the index over the last year?

Let’s be blunt: the Mozscape index has had a hard time this year. We’ve been slow to release, and the size of the index has jumped around.

Before we get down into the details of what happened, here’s the good news: We’re confident that we have found the underlying problem and the index can now improve. For our own peace of mind and to ensure stability, we will be growing the index slowly in the next quarter, planning for a release at least once a month (or quicker, if possible).

Also on the bright side, some of the improvements we made while trying to find the problem have increased the speed of our crawlers, and we are now hitting just over a billion pages a day.

We had a bug.

There was a small bug in our scheduling code (this is different from the code that creates the index, so our metrics were still good). Previously, this bug had been benign, but due to several other minor issues (when it rains, it pours!), it had a snowball effect and caused some large problems. This made identifying and tracking down the original problem relatively hard.

The bug had far-reaching consequences…

The bug was causing lower-value domains to be crawled more frequently than they should have been. This happened because we crawled a huge number of low-quality sites for a 30-day period (we’ll elaborate on this further down), and then generated an index with them. In turn, this raised all these sites’ domain authority above a certain threshold where they would have otherwise been ignored, when the bug was benign. Now that they crossed this threshold (from a DA of 0 to a DA of 1), the bug was acting on them, and when crawls were scheduled, these domains were treated as if they had a DA of 5 or 6. Billions of low-quality sites were flooding the schedule with pages that caused us to crawl fewer pages on high-quality sites because we were using the crawl budget to crawl lots of low-quality sites.

…And index quality was affected.

We noticed the drop in high-quality domain pages being crawled. As a result, we started using more and more data to build the index, increasing the size of our crawler fleet so that we expanded daily capacity to offset the low numbers and make sure we had enough pages from high-quality domains to get a quality index that accurately reflected PA/DA for our customers. This was a bit of a manual process, and we got it wrong twice: once on the low side, causing us to cancel index #49, and once on the high side, making index #48 huge.

Though we worked aggressively to maintain the quality of the index, importing more data meant it took longer to process the data and build the index. Additionally, because of the odd shape of some of the domains (see below) our algorithms and hardware cluster were put under some unusual stress that caused hot spots in our processing, exaggerating some of the delays.

However, in the final analysis, we maintained the approximate size and shape of good-quality domains, and thus PA and DA were being preserved in their quality for our customers.

There were a few contributing factors:

We imported a new set of domains from a partner company.

We basically did a swap with them. We showed them all the domains we had seen, and they would show us all the domains they had seen. We had a corpus of 390 million domains, while they had 450 million domains. A lot of this was overlap, but afterwards, we had approximately 470 million domains available to our schedulers.

On the face of it, that doesn’t sound so bad. However, it turns out a large chunk of the new domains we received were domains in .pw and .cn. Not a perfect fit for Moz, as most of our customers are in North America and Europe, but it does provide a more accurate description of the web, which in turn creates better Page/Domain authority values (in theory). More on this below.

Palau, a small island nation in the middle of the Pacific Ocean.

Palau has the TLD of .pw. Seems harmless, right? In the last couple of years, the domain registrar of Palau has been aggressively marketing itself as the “Professional Web” TLD. This seems to have attracted a lot of spammers (enough that even Symantec took notice).

The result was that we got a lot of spam from Palau in our index. That shouldn’t have been a big deal, in the grand scheme of things. But, as it turns out, there’s a lot of spam in Palau. In one index, domains with the .pw extension reached 5% of the domains in our index. As a reference point, that’s more than most European countries.

More interestingly, though, there seem to be a lot of links to .pw domains, but very few outlinks from .pw to any other part of the web.

Here’s a graph showing the outlinks per domain for each region of the index:

TQu--jaKCoqQLiRknNQw42R7GeMWfkuuKmDCOBUTmZ2Eg6FW1grq3z6oBJMZm_wItHmOD_K7UDicMgq_8OkLVnjLKDNxoRMfgU20B2ymlQK7eueKqIAcY3wsqfJizRwo7hnt7Yw2jA

China and its subdomains (also known as FQDNs).

In China, it seems to be relatively common for domains to have lots of subdomains. Normally, we can handle a site with a lot of subdomains (blogspot.com and wordpress.com are perfect examples of sites with many, many subdomains). But within the .cn TLD, 2% of domains have over 10,000 subdomains, and 80% have several thousand subdomains. This is much rarer in the North Americas and in Europe, in spite of a few outliers like WordPress and Blogspot.

Historically, the Mozcape index has slowly grown the total number of FQDNs, from ¼ billion in 2010 to 1 billion in 2013. Then, in 2014, we started to expand and got 6 billion FQDNs in the index. In 2015, one index had 56 billion FQDNs!

We found that a whopping 45 billion of those FQDNS were coming from only 250,000 domains. That means, on average, these sites had 180,000 subdomains each. (The record was 10 million subdomains for a single domain.)

Chinese sites are fond of links.

We started running across pages with thousands of links per page. It’s not terribly uncommon to have a large number of links on a particular page. However, we started to run into domains with tens of thousands of links per page, and tens of thousands of pages on the same site with these characteristics.

At the peak, we had two pages in the index with over 16,000 links on each of these pages. These could have been quite legitimate pages, but it was hard to tell, given the language barrier. However, in terms of SEO analysis, these pages were providing very little link equity and thus not contributing much to the index.

This is not exclusively a problem with the .cn TLD; this happens on a lot of spammy sites. But we did find a huge cluster of sites in the .cn TLD that were close together lexicographically, causing a hot spot in our processing cluster.

We had a 12-hour DNS outage that went unnoticed.

DNS is the backbone of the Internet. It should never die. If DNS fails, the Internet more or less dies, as it becomes impossible to lookup the IP address of a domain. Our crawlers, unfortunately, experienced a DNS outage.

The crawlers continued to crawl, but marked all the pages they crawled as DNS failures. Generally, when we have a DNS failure, it’s because a domain has “died,” or been taken offline. (Fun fact: the average life expectancy of a domain is 40 days.) This information is passed back to the schedulers, and the domain is blacklisted for 30 days, then retried. If it fails again, then we remove it from the schedulers.

In a 12-hour period, we crawl a lot of sites (approximately 500,000). We ended up banning a lot of sites from being recrawled for a 30-day period, and many of them were high-value domains.

Because we banned a lot of high-value domains, we filled that space with lower-quality domains for 30 days. This isn’t a huge problem for the index, as we use more than 30 days of data — in the end, we still included the quality domains. But it did cause a skew in what we crawled, and we took a deep dive into the .cn and .pw TLDs.

This caused the perfect storm.

We imported a lot of new domains (whose initial DA is unknown) that we had not seen previously. These would have been crawled slowly over time and would likely have resulted in their domains to be assigned a DA of 0, because their linkage with other domains in the index would be minimal.

But, because we had a DNS outage that caused a large number of high-quality domains to be banned, we replaced them in the schedule with a lot of low-quality domains from the .pw and .cn TLDs for a 30-day period. These domains, though not connected to other domains in the index, were highly connected to each other. Thus, when an index was generated with this information, a significant percentage of these domains gained enough DA to make the bug in scheduling non-benign.

With lots of low-quality domains now being available for scheduling, we used up a significant percentage of our crawl budget on low-quality sites. This had the effect of making our crawl of high-quality sites more shallow, while the low-quality sites were either dead or very slow to respond — this caused a reduction in the total number of actual pages crawled.

Another side effect was the shape of the domains we crawled. As noted above, domains with the .pw and .cn TLDs seem to have a different strategy in terms of linking — both externally to one other and internally to themselves — in comparison with North American and European sites. This data shape caused a couple of problems when processing the data that increased the required time to process the data (due to the unexpected shape and the resulting hot spots in our processing cluster).

What measures have we taken to solve this?

We fixed the originally benign bug in scheduling. This was a two-line code change to make sure that domains were correctly categorized by their Domain Authority. We use DA to determine how deeply to crawl a domain.

During this year, we have increased our crawler fleet and added some extra checks in the scheduler. With these new additions and the bug fix, we are now crawling at record rates and seeing more than 1 billion pages a day being checked by our crawlers.

We’ve also improved.

There’s a silver lining to all of this. The interesting shapes of data we saw caused us to examine several bottlenecks in our code and optimize them. This helped improve our performance in generating an index. We can now automatically handle some odd shapes in the data without any intervention, so we should see fewer issues with the processing cluster.

More restrictions were added.

  1. We have a maximum link limit per page (the first 2,000).
  2. We have banned domains with an excessive number of subdomains.
    • Any domain that has more than 10,000 subdomains has been banned…
    • …Unless it is explicitly whitelisted (e.g. WordPress.com).
      • We have ~70,000 whitelisted domains.
    • This ban affects approximately 250,000 domains (most with .cn and .pw TLDs)…
      • …and has removed 45 billion subdomains. Yes, BILLION! You can bet that was clogging up a lot of our crawl bandwidth with sites Google probably doesn’t care much about.

We made positive changes.

  1. Better monitoring of DNS (complete with alarms).
  2. Banning domains after DNS failure is not automatic for high-quality domains (but still is for low-quality domains).
  3. Several code quality improvements that will make generating the index faster.
  4. We’ve doubled our crawler fleet, with more improvements to come.

Now, how are things looking for 2016?

Good! But I’ve been told I need to be more specific. :-)

Before we get to 2016, we still have a good portion of 2015 to go. Our plan is stabilize the index at around 180 billion URLs for the end of the year and release an index predictably every three weeks.

We are also in the process of improving our correlations to Google’s index. Currently our fit is pretty good at a 75% match, but we’ve been higher at around 80%; we’re testing a new technique to improve our metrics correlations and Google coverage beyond that. This will be an ongoing process, and though we expect to see improvements in 2015, these improvements will continue on into 2016.

Our index struggles this year have taught us some very valuable lessons. We’ve identified some bottlenecks and their causes. We’re going to attack these bottlenecks and improve the performance of the processing cluster to get the index out quicker for you.

We’ve improved the crawling cluster and now exceed a billion pages a day. That’s a lot of pages. And guess what? We still have some spare bandwidth in our data center to crawl more sites. We plan to improve the crawlers to increase our crawl rate, reducing the number of historical days in our index and allowing us to see much more recent data.

In summary, in 2016, expect to see larger indexes, at a more consistent time frame, using less historical data, that maps closer to Google’s own index. And thank you for bearing with us, through the hard times and the good — we could never do it without you.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don’t have time to hunt down but want to read!


Moz Blog

Posted in IM NewsComments Off

Bing Now Lets You Play Pong In Their Search Results

Bing added a game to their search results, play Pong against Bing by searching for [pong].

The post Bing Now Lets You Play Pong In Their Search Results appeared first on Search Engine Land.



Please visit Search Engine Land for the full article.


Search Engine Land: News & Info About SEO, PPC, SEM, Search Engines & Search Marketing

Posted in IM NewsComments Off

Advert