Simple_Press - Creating Custom Importers

Introduction

Simple:Press has an importer plugin that imports data from a variety of legacy forum systems including bbPress, wpForo and Asgaros.

One of the objectives when we created the importer was to make it easy for your average developer to quickly create an importer for any forum. To that end we included a framework that, once understood, makes it easy to create a new importer process.

An experienced developer should be able to create an importer that imports topics and posts in three days or less.

In this article we’ll describe how this framework works and walk you through the process of creating your own.

The Heart of An Importer

Lets explore the elements at the heart of a Simple:Press importer.

Each custom importer is a single file located in the importers folder under the Simple:Press importer plugin. The full path-name on your server is likely something such as your-server-root/wp-content/sp-import/importers.

When you open up one of these files you will notice a number of different sections:

  • Source DB Settings
  • MISC Definitions
  • Table/Data Mapping (with one subsection each for users, groups, forums topics and posts)
  • Form

Additionally, the top of the file has a formal structured header provides some key information to the Simple:Press importer engine.

The heart of the importer is the TABLE/DATA Mapping section. This is where you’ll tell the importer the names of the tables and fields that you’re pulling data from. If need be, you can provide custom SQL commands in this section to join tables to collect additional information as needed.

With this high-level view out of the way, lets get started with writing a custom importer for Simple:Press!

Prepare Your Importer File

The first thing we need to do is to make a copy of one of the existing importer files. For the purposes of this article we’ll make a copy of the wpforo.php file under the importers folder. Lets call this new file wpforoarticle.php.

Once you have made a copy of the file, lets change some key information in it:

In the header section change the name of the importer, the filename and the tag. Your new section should look something like this:

Then, scroll down to the FORM section at the bottom of the file and change any WPFORO text to text that references our new importer name: WPforo Article. This is text that will appear on the screen so its best to change it to something that references your old forum name.

In the image below you can see some of the places in the form where we’ve made a change:

The form contains two important pieces of information that must be changed:

  • The Form Name
  • A parameter that is sent to the sp-importsetup.php file.

At this point, if you access your importer plugin you should see the new option on the main screen:

If you’d like to change the logo that is shown on the screen you can add the logo file to the importers/logos folder. Then, reference the file name in the wpforoarticle.php header:

Data Mapping

Now, lets tell the importer where to find the data it needs. For the purposes of this article we’ll assume that your old forum system has separate tables for:

  • Forum Definitions
  • Topics
  • Posts

Additionally, we’ll assume that the data is in a standard MYSQL database located somewhere that is accessible from your WordPress server.

With these assumptions in place, there are no changes necessary to the Source DB Settings and Misc Definitions Sections.

So, the first section that needs to change is the USERS Table Mappings.

The first line is set to true so that the import knows that it should attempt to import users.

The second line is the name of the table in your old forum software where the users information is located.

Two lines below that are the field names from that table that will map to Simple:Press field names. So you need to tell Simple:Press which fields in the old forum software users table contains the user id, login name, password etc. If a field is not available, just leave it blank with two single quotes.

Tip: If you are importing data from another WordPress plugin, chances are that they are using the WordPress users table to store users information. In this case there should be no changes to the user section.

With the users section out of the way, lets move on to the FORUMS section.

FORUMS Map

The FORUMS map is the list of forums in your old forum software. Most forum software tend to allow for multiple forums so there’s usually a table that contains the name of the forum, its id and other related information. We call that the FORUMS table and we will map that data to the Simple:Press SFFORUMS table.

Just like with the user information we pulled our in the prior section, you need to provide the name of the table that holds the forum information and the fields that contain the individual pieces of data:

The most important piece of information here is the forum id. But most forum software also has the forum title and a description that you can pull out.

You can leave the Group ID and Group Sequence fields alone – its usually not easy to pull that data out of most legacy software and many of them simply don’t support groups. Once the forums are imported you can quickly create new groupings inside the Simple:Press admin dashboard.

TOPICS and POSTS Maps

By now you’re probably seeing the pattern. So you just need to fill out the information for the TOPICS table and the POSTS tables.

At this point you have provided all the information that the importer engine needs. So you can go ahead and try out your import!

Advanced Topics

Custom Views

You’re probably thinking – this is too simple, right? However, you’d be surprised at the number of legacy forum software where this simple process works.

There are, of course, some cases where the table structure of the software is complex and you need to pull data from more than one table in order to get everything you need for a particular map.

We do handle this situation of course but you will need to put your mad SQL skills to work. A couple of importers where this came in handy were the bbPress importer and the Asgaros Forum importer.

Specifying Custom Views

You might have noticed that in each of the MAPPING arrays, there are two elements that we haven’t used yet – the SELECT and WHERE elements. This is where you would write your SQL that joins tables.

Effectively your SELECT statement creates a SQL VIEW that simulates a table. The names of the columns in that view are used to populate the rest of the map.

Below are a couple of examples from our bbPress importer. bbPress stores its data in the WordPress POSTS table so to extract just the forum data requires a couple of SQL statements.

For a more complex SELECT statement, check out the ASGAROS importer where we had to join a couple of tables:

Custom Post Processing

If you need to massage the data or do some post processing after the basic import is complete, take a look at the engine/spimport.php file. This is where all of the import logic actually resides.

There are no action or filter hooks available yet in this file but we do plan on adding them at some point. So far no one has asked us for them but once we get our first request we’ll make sure they get in there.

Wrapup

As you can see, its not that difficult to write a Simple:Press importer once you know the process. Most of the time is usually spent in testing and in researching the table structure of the legacy forum software. An experienced developer should be able to create an importer in three days or less as long as the table structure of the legacy forum software isn’t too obscure.

If you’d like us to write a custom importer for you just drop us a note via our contact form and we’ll get you a custom quote.

If you do write an importer of your own, we’ve love to see what you did so feel free to send us your importer file.

And for any other questions drop us a note in our Support Forums!

Did you find this article useful? Or do you use Simple:Press today?  If so, please support Simple:Press with a review on WordPress.org.  Every review goes a long way towards bringing other users on board!

Click here to review Simple:Press on wordpress.org

More From Simple:Press

User Story: Keeping The Brooklyn Triathlon Club Connected

June 13, 2019

This article is the third in a series of “user stories” where our users describe how they built their sites using Simple:Press. In this story you will learn how Vicky Wei built out a WordPress membership site using Simple:Press and the WishList Member plugin. Introduction The Brooklyn Triathlon Club has been around since 2004 and…

Read More

2nd Quarter 2019 Progress Report: New Plugins and Recent Updates

May 21, 2019

Over the last couple of months we’ve been busy making tweaks and enhancements to a number of our plugins and themes. The result of this work can be see in the following updates: New Simple:Press Core Release Version 6.2.0 Simple:Press is now available in the WordPress.org repository. This release includes the following changes: New: SVG…

Read More

How We Do Backups

May 10, 2019

Introduction Occasionally we write about how we run out business – the tools and technology we use. This article is one of those and it covers a very important topic that many sites and business don’t pay enough attention to. Hopefully by reading this you’re reminded to take another look at how your backups work…

Read More

Move To Semantic Versioning Complete

May 3, 2019

If you’ve never heard the term “semantic version”, don’t feel bad – most end users of software have never heard the term either. But more and more, software is using the concept to govern the release numbers that are assigned to new public releases. In the case of Simple:Press, we’ve completed our to move to…

Read More