sm-chrome-dev

SourceMogul Anysite will grab the information off the page you are visiting.

This repository contains both the client and the server for the extenstion to work.

Client

After you have cloned the repo copy the config.js.ansible to config.js and edit with your dev settings

Webpack is used to bundle up all the relevent files need to run the Chrome Extension.

To build the system for development first run npm install in the root folder, copy the config.js.ansible file to config.js, add your dev environment settings and then npm run dev. This will create you a build folder which you can the load into Chrome.

npm run dev, watches the files so any chnages you make are automatially show in the extesntions. The only files that do not do that, are the manifest, backgroand page and any content scripts. You have to click refresh in the extenstion page to reload these files before you see any changes.

React App Break down

In the manifest you can see that under browser_action it list a file called popup.html, this is a standard html file that loads some css creates a div with the id of root and loads a javascript file called app.js. The app.js file is the react app and is loaded into the id root div when it is run.

In the webpack file you can see under entry a key called app and a value called App.jsx. This is what bundles up all the files into app.js file.

App.jsx

The entry point to the Chrome Extension, here we setup all the boiler plate ccode for Redux and then render the aoo in the div with the id of root. It's not until you get to main.jsx until things get more exciting.

We are using redux-persist-chrome-storage, so when you save an item into Redux is is also stored into chrome storage, if we did not do this, then next time ethe extension loads we would start with no data.

There is no way of seeing the chrome storage in a nice way. You can only see the storage by opening the background page from the Extensions page and in this console run the following command:-

chrome.storage.local.get(function(result){console.log(result)})

main.jsx

Here we check if we have a user key in our redux store. If we have no key we display the enter key page KeyInput. If we do have a key we check load CheckPage.

CheckPage

In the constructor of this React component we call a local function called setPageState. This grabs the url of the current active tab and post to the server. The response you get will either be OK_DOMAIN and the type of pagination the domain uses, or WRONG_DOMAIN and not pagination type.

The response is then saved into Redux and the page will be rendered again and will either run UnknownDomain if we have WRONG_DOMAIN or SearchDomain is we get OK_DOMAIN.

UnknownDomain

This is a simple page which says we don't know how what to do with this domain

SearchDomain

Search Domain gives you the option to start searching the page. When you click nthe button we then post the info to the server which will create a new jobId. This jobId is then saved in Redux and we change the state of the search to IN_PROGRESS. The current page is reloaded so the content page sendHtml.js can be run again.

Content Page - sendHtml.js

The content script is run on every page the users vists when they have the chrome extension installed (we might be able to change this by just using the active tab permission, but for now this is how it works).

sendHtml script is able to access the chrome storage and checks to make sure the search state is IN_PROGRESS and the current window.location.host matches the domain we started the search with. If they we do grab the html and post it down to the server so be procesed and saved into mongodb html collection along with the jobId.

searchInProcess

CheckPage renders the searchInProgress page when the search state is IN_PROGRESS. The first thing this component does is get the saved job info from the database and displays it in Redux.

From this page you can pause, discard or Compelte Search.

Pause will display the Discard Page?!? (Should prbably be pause page)

Complete Search resets the Redux store and posts to the server telling it to process the resulst.

Starting the Server

#!bash

cd server
npm install
cp config.js.ansible config.js
edit config.js to add the right vars
pm2 start --name=sm-chrome server.js 

Setup of React and Webpack

This is a great tutorial on how to setup react and webpack up:-

https://dev.to/iamismile/how-to-setup-webpack-and-babel-for-react-59ph

Test for Alaa