Last week I received an email from a new reader:
I want to create a site like The Daily Bolt, but for [a different niche]. How did you set everything up?
While part of me would love to protect the details of how I created The Daily Bolt, a site I created to aggregate Genesis and WordPress news in one place, I really can’t justify that since anyone can perform Google searches and find the same information I did. Instead, I figured I’d just share it and see what kind of cool new sites crop up.
How The Daily Bolt works
As of this writing, The Daily Bolt pulls in RSS feeds from 33 websites related to Genesis, WordPress, and/or freelancing. When any of those sites has new content, a post is created and published on my site automatically.
If you visit the site and click on a post title, you’ll see that you’re taken directly to the original article. This is intentional, as I didn’t want to step on toes or take traffic away from the writers who are putting out great news articles and tutorials. Aggregating other people’s work has the potential to cause problems, so I made sure I was doing it in a friendly way.
How to aggregate posts from other WordPress sites
I originally used a plugin called RSS Multi Importer to bring in all the feeds I wanted, but the plugin stopped working last week and mysteriously vanished from the WP repo. I switched to RSS Post Importer yesterday and so far it does what I need it to do. *knocks on wood*
Caveat: If you want to pull in full posts, RSS Post Importer requires a paid integration to do that. The free version only allows excerpts. While this was fine for The Daily Bolt, it may not work for the type of aggregator site you want to build, so be prepared to spend a little if you need advanced functionality.
Once you’ve installed the plugin, you’ll visit Settings > RSS Post Importer in your dashboard. You can add new feeds at the top and adjust settings at the bottom. Here’s what the Add New Feed options look like (click to enlarge):
The only things I changed were the feed name and URL, max posts per import (I chose ten since I was rebuilding what I lost from the old plugin), and the category. After adding all the feeds, I clicked Save and Import at the top right of the screen to start bringing content back in.
In the settings section at the bottom, I changed the frequency to hourly and made sure the links will be nofollow. That’s basically all it took to repopulate The Daily Bolt with all the news goodness it had before my previous plugin died.
Optional: Setting up automatic email campaigns.
One of the hallmarks of The Daily Bolt is a daily digest email sent out via MailChimp. Those who don’t want to visit the site directly can sign up for a single daily email with all the day’s headlines, which also click through to the original articles (thanks to a redirect, which I explain below). MailChimp’s RSS to email feature makes this super easy, and now that it’s set up, I don’t have to do anything to maintain it except making sure the emails go out when they should.
Warning: Do not spam your list. Several people have asked how I plan to monetize the email list, and the simple answer is, I’m not. I promised people one email per day max (none if there are no new posts that day) and that’s all I’ll ever do with it. Way too often, I’ve signed up for similar lists and been bombarded with sales junk and “special deals” as a result. If people trust me enough to give me their email addresses, I’m going to be responsible. The end.
Now that I’m off my soapbox, that doesn’t mean you can’t choose to set up your email list differently than I did. I’m just saying, if people are signing up for one thing, don’t give them another. Use other methods (or other lists) to market to people – aggregator sites are (usually) about sharing great content.
Other steps to make your life easier
Redirecting single posts to the original article.
When I first started The Daily Bolt, visitors had to click through to the single post screen on my site, then click a link to go to the full article. Lame, right? After a ton of searching, I found a handy code snippet from @nathanrice to redirect all single posts to the URL in RSS Post Importer’s custom field (please note that you’ll need to alter the code if you use a different plugin to pull in RSS feeds):
Opening single posts in a new tab.
I also added a filter to the Genesis post title to add target=”_blank” to all entry title links, which causes each post to open in a new tab. This allows visitors to come back to The Daily Bolt tab to look at other recent posts.
Increasing the number of posts per page.
The Daily Bolt uses a custom Genesis child theme that displays posts in a 3-column grid. I wanted to set the posts per page to a multiple of 3, so I added this to functions.php as well:
Getting feedback before you launch.
Before I launched The Daily Bolt, I initially decided to reach out to each blogger directly to make sure it was okay to include his/her posts. However, after four people in a row said, “It links to my site and you’re only showing an excerpt? GO RIGHT AHEAD!” I decided to skip the others, with a policy to immediately remove any feeds if the blogger asked me to. (For the record, so far no one has complained about free traffic.)
That said, it’s still a good idea to talk to people first. I know nearly every person whose feed I’m using, so my situation might be a little different from yours. I didn’t just start bringing in random RSS feeds (for the most part) and I’m not grabbing full posts – those things are important.
Wrapping it all up
It’s really not complicated to create a WordPress aggregator site just like The Daily Bolt. The exact steps you take will depend on what theme you’re using, what other features you plan to offer, whether you’re also including original content, and a slew of other factors, but the basics above will help you set up a resource that other people will appreciate.
The most important step for any project is planning – making sure you know what you want the end result to be and working toward that goal. In my case, I wanted to create a resource for my own use (or laziness, whichever), and if other people used it, great! If not, that’s okay, too. The Daily Bolt was created with that in mind, though there are TONS of use cases for a similar site with your own features and personal touches.
Have you created a site with aggregated RSS feeds, or do you have any plans to build one? I’d love to check out what you’re working on!
Elsa M says
LOVE The Daily Bolt and thank you for this tutorial. I’m following the redirecting code using the same RSS plugin (for a client) but is not working for me 🙁 I must be getting stuck somewhere!
Are you adding the code to functions.php or single.php and where did you add the custom field?
Thanks again!
Andrea Whitmer says
Hi Elsa,
All the code I added went in functions.php. The custom field is one generated automatically by the RSS plugin, so the redirect function is just pulling from that information.
Elsa M says
Hi Andrea,
Thanks for the reply!
I’ll keep working on it, I must be missing something.
kwad says
Great tips. I was looking for a way to aggregate tech news onto my site launching next year. I am now looking forward to putting these steps into action when i get time over the next week.
Kwad.
Andrea Whitmer says
Be sure to check back in and share your URL – I look forward to seeing the site! 🙂
Sam says
Greta post, thanks! I need that code for opening links to the source… but I am not using Genesis… any help?
Andrea Whitmer says
The only part of that specific to Genesis is
genesis_get_custom_field
– you could use a different method to get the field: https://codex.wordpress.org/Custom_Fields#Getting_Custom_FieldsSam says
Hmmm… I tried swapping it out for get_post_meta but nothing happens at all.
Andrea Whitmer says
You might try reaching out to the author of your theme to see if there is a specific function or method you need to use. I haven’t used anything but Genesis for several years so unfortunately I’m probably not a lot of help.
Victor Dominic says
Hello Sam, where you able to solve this,
If yes how did you do it?
Thanks.
Clark says
Thank you for a wonderfully written article. It was quite helpful to me in setting up my own aggregator site. Instead of the php code I used the plugins ‘WP-External-Links’ and ‘Autoblog’ from WPMU Dev; so far everything is working beautifully. Thanks again for sharing with the community.
Andrea Whitmer says
Thanks so much for sharing! I really like the layout of the site – very easy to navigate. 🙂
Jean says
Hey Andrea, we’ve been in touch already, I love the daily bolt and it inspired me to create a similar site for WordPress news in general, voila WP News Desk. I built it with WP RSS Aggregator and we also shared a tutorial on how we built it over at WP Mayor.
Andrea Whitmer says
Thanks for sharing – looks great!
Dizzy says
Hi… I created an aggregator site after being inspired by the Daily Bolt. How do you minimise your disk space? My site is eating up my disk space like crazy!
Andrea Whitmer says
I use a plugin called Bulk Delete (I think?) to delete everything older than 90 days. I have a reminder in my phone every 3 months so there is never anything older than that. Otherwise the posts build up like crazy, as do their related images. I have to remove those manually via FTP since removing the posts doesn’t clear the media library.
Dizzy says
Thank you so much. You are a life saver!
Devin says
Hello,
Great post I really enjoyed it. I am currently looking at creating an aggregator for personal research. I am looking at having feed from 600 websites. I do not know that I would need to actually create a webpage? I essentially would like to do the exact same thing as you but I do not need it public for others. Could I create an aggregator similar to the one you created without a webpage?
Andrea Whitmer says
If you don’t need a website, I’d probably just use an RSS reader like Feedly.
Benneth says
Hi Andrea,
Am planning to build a news aggregator with the WPRSSAgregator plugin, I would be pulling in contents from 70 sites (feed sources) every 30 minutes or 1 hour, and I plan to host the site on siteground Gogeek plan would that slow my site?
Andrea Whitmer says
Hi Benneth,
I think you’d probably do okay with that number of feeds. The plugin I use runs one feed at a time, though it does update them all in tandem. I don’t notice any slowing on the front end while this is happening, and it runs pretty quickly.
Victor Dominic says
Hello Andrea, thanks for the post, it was really explicit. I want to know if it’s possible to add thumbnails to your site using this plugin?
Andrea Whitmer says
Yep, you sure can! I opted not to show them in the custom theme I built for The Daily Bolt, but you can bring them in and display them if you choose.
Victor Dominic says
How to Redirect to the source site
To those having problem redirecting to the source site without using the genesis theme, you can use the code snippet below.
//* Redirect posts to original URL
add_action( ‘template_redirect’, ‘nabm_original_post_redirect’ );
function nabm_original_post_redirect() {
if ( ! is_singular() ) {
return;
}
if ( $url = get_post_meta( get_the_ID(), ‘rss_pi_source_url’, true ) ) {
wp_redirect( esc_url_raw( $url ), 301 );
exit;
}
}
Remember to place it in your functions.php
This is an ancient article,
so comments are now closed.