Oct 28, 2020

How to develop DEX8 Skript

Introduction

When developing DEX8 Skripts, it is important to consider that the code should be executed in a DEX8 Worker environment. The DEX8 Worker is designed for NodeJS, so every Skript should be written in vanilla JavaScript. The Worker includes installed NPM packages such as Puppeteer, Cheerio, Bluebird, Mongoose, etc., which are defined in package.json.

There are two ways to create a DEX8 Skript:

Web Panel - Login into the Web Panel and code your skript using your web browser.
DEX8-CLI - Install dex8-cli and develop the skript using your favorite code editor.

The second method is faster and more flexible, so this tutorial will focus on it.

Prerequisites

Prior Knowledge Prerequisites

Modern JavaScript ES6+ (ES2015)
JavaScript promises
JavaScript async/await
NodeJS 14+ API (https://nodejs.org/en/docs)
Chrome Puppeteer (https://pptr.dev)
Git version control

Technical Prerequisites

Installed NodeJS 14+ on Linux, Windows, or Mac
Installed dex8-cli npm ($ sudo npm install -g dex8-cli)
Visual Studio Code Editor (https://code.visualstudio.com)
Installed git
Installed ESLint $ sudo npm install -g eslint (https://eslint.org/docs/user-guide/getting-started)

DEX8 CLI

The dex8-cli npm package is not mandatory, but it is highly recommended. It speeds up development and improves code readability. Additionally, powerful libraries like @mikosoft/functionflow, @mikosoft/mongofast, @mikosoft/cookie-storage, @mikosoft/echo, and @mikosoft/httpclient-node can be used.

Install globally using:

$ npm install -g dex8-cli

Init New Skript

To create a skript folder with initial files, use:
$ dex8 init

For example, running $ dex8 init will create a skript folder with basic files, which can be modified later.

The Basic Files

.editorconfig - Editor settings (https://editorconfig.org)
.eslintrc - Maintains code quality
.gitignore - Git ignore rules
package.json - Defines NPM packages (modifiable for additional dependencies)
manifest.json - Contains DEX8 Skript metadata
howto.html - Instructions on using the skript
main.js - Main execution file
input.json - Contains input data for the skript
inputSecret.json - Stores confidential data such as passwords and tokens

Start Skript

To test the skript from the terminal, use:
$ dex8 start -i input.json

If input is not required, simply use:
$ dex8 start

Upload Skript

If the skript runs without errors, upload it to the DEX8 system:
$ dex8 upload

Once uploaded, log in to the Web Panel and deploy the skript to one or multiple DEX8 Workers.

Puppeteer Installation

Puppeteer is an essential library for web automation.

Installation

Install Puppeteer without Chromium for faster installation:
$ npm install puppeteer-core

Manifest File

The manifest.json file contains metadata for the skript, such as "title", "description", "thumbnail", "category", and "environment". Modify all fields except "howto" and "files", which are automatically updated upon skript upload.

{
  "title": "mySkript",
  "description": "This is my first DEX8 Skript.",
  "thumbnail": "https://cdn.dex8.com/img/shop_products/skripts3.png",
  "category": "general",
  "environment": "nodejs",
  "worker_response_timeout": 180000
}

Howto File

The howto.html file should contain detailed documentation and usage instructions for the skript.

<h1>mySkript</h1>
<p>This is my first DEX8 Skript.</p>

Function Files

The main.js file references various functions for processing and extracting data. Example workflow functions:

Connect to the database
Open the homepage
Login
Navigate to a page
Extract data
Save data to the database
Logout

Input & Input Secret Files

Input files store initial data necessary for the skript. Any file named "input.json" or "inputSecret.json" will be recognized automatically.

Start and Upload Skript

Use $ dex8 start -i input.json to test and $ dex8 upload to deploy your skript.

Now, sit back and enjoy the results!
😊

Tutorials