Introduction

The Importer tool is used to import comments for a publisher who has chosen to go live with Insticator but was using another vendor’s commenting tool earlier.

The authors, comments, and threads of the previous vendor will be ported into the Insticator commenting unit (barring exceptions due to moderation or invalid data like empty threads).

• The import process has to be initiated by an admin user
• Please reach out to your respective Account Manager to start the Import process.

---

Vendor Support

🚧

Disqus

• When using the Disqus API we require an API Key.
• If the account is "Basic" we are restricted by Disqus on the amount of data we can ingest and how quickly we can ingest it (date ranges should be shorter in this case).

📘

Non-Supported Vendors & Formats

• Some vendors have plugins or other variables that can affect the format of the data.
• We cannot support all possible variations.
• We can however workaround these issues.

---

Notable Features

• The option to abort an ongoing import.
• The option to upload a file directly from the system - no need to upload to S3 anymore.
• Logs are displayed on the page to aid in debugging.
• An email will be sent to the admin once the import is completed.
• Engineers are sent a slack notification if the import fails.
• New security logic
• Enhanced UI and UX
• Error Handling to enable resuming the process from where it broke rather than starting over.

---

How to Guide

Below please find step-by-step instructions on how to use the tool:

1. Go to Insticator Publisher Portal 2.0→Commenting→Importer Tool.

2. Click on ‘New Import’. On clicking the ‘New Import’ button, an option to configure the import will be shown.

3. You need to select the ‘Vendor’ first. Based on your selection the remaining options will appear:

   Vendor Disqus:
   Disqus API - Input the API Key and Forum Name, select the ‘Disqus plan’ that you have subscribed for, add the date range and click on start import.
   Disqus Plan:
   Basic: Limited to 1000 API requests/hour
   Pro: No such limitation
   Disqus XML File - Click on the ‘Select File’ option and upload the XML file from your system.

   Vendor Vuukle:
   Vuukle XML/JSON File - Click on the ‘Select File’ option and upload the XML/JSON file from your system.

   Vendor OpenWeb:
   OpenWeb CSV File - Click on the ‘Select File’ option and upload the CSV file from your system.

   Map URLs option - checks every URL from the export file by going to a given page and checking the current URL - if it’s different from the one in the export file, it will override the imported one with the new URL value.

   Use article as extPageID option - It allows us to properly map commenting threads imported from “Mapping URLs” since it would be pretty time and resource consuming to check every URL

   Note:
   a) Please keep the importer tool tab of your browser open while the file upload is complete.
   b) Make sure the file is < 30 MB or < 500K rows.

4. Once the import has started, you will be able to see the ongoing import process in a table as shown below:

5. The table shows the Import Source, Import Duration, Rough ETA, and Authors, Threads, Comments with the following statuses:

   a) Pending: Data that is getting parsed from files OR scraped from the API and stored into the intermediate DB.
   b) Prepared: Validated comments in the intermediate DB that are added to the queue to get imported into the prod DB.
   c) Skipped: Data in the intermediate DB that will not be imported into the prod DB due to their statuses (Declined, Deleted)
   d) Empty: Threads that don’t have any comments and won’t be created.
   e) In Process: Data that is getting imported into the prod DB at the moment.
   f) Postponed: Comment replies that don’t have root comments, which have been imported already/exist in our DB. Postponed comments persist inside of the intermediate DB until we will get their “reply to”/”root comment” so we can import them and display.
   E.g.The root comment was posted 10 days ago, but the reply was added just yesterday and we scraped data only from yesterday. Then tried importing that reply, but haven’t scraped the root comment yet and put it on hold. And only once we have that root comment - we’ll import that reply from “postponed” .
   g) Retrying: Data that was postponed is being retried
   h) Failed: This particular author,thread or comment has failed and cannot be imported
   i) Ported: All data that has been already imported into the prod DB.
   j) Total: Total of all rows (statuses).

6. The Status can take the following values:
   a) Parsing: Data is being transferred into temporary DB
   b) Running: Data is being transferred from temp DB to Production
   c) Completed: The import process has been completed
   d) Failed Error: The import process has failed. Please start a ‘New Import’
   e) Canceled: The import was canceled

7. The Documentation button at the top right hand corner directs you to this page.

8. The Actions column contains two buttons:
   a) ‘X’ - To abort the ongoing import process. This button can only be used when the status is ‘Parsing’
   b) ‘Preview’ button - This option enables the user to see previews of the threads that are being imported. This button can only be used when atleast 1 thread has been ported successfully.
   The preview button opens into a new tab as shown below:
   

9. On the top right you can input the number of ‘top threads’(threads with most comments) you want to preview and click on ‘Go’
   The Original URL sections shows the original comments from the URL we have exported the comments.
   The Demo URL shows the Imported comments to check if everything was imported perfectly.

❗️

Important Points to Remember

• Only 1 import can be run at a time.
• The importer tool tab needs to be kept open while the file is uploading.
• The ‘X’ option can only be used while the status is ‘In progress’.