This Mock Data Scheme is part of Usecase UI’s best practice. We develop this scheme to help frontend developers promoting developing efficiency and code quality, which is also an important method to practice the concept of ‘frontend-backend separation’.
1. Pre-optimizated Condition
The Usecase-UI project uses Angular as framework. Before building the mock data scheme, we have already optimized the structure of this project in according to the best practice of
building Angular project. After optimizing, the project structure is as follow:
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
├── e2e ├── src │ ├── app │ │ ├── core │ │ │ ├── models │ │ │ └── services │ │ ├── shared │ │ │ ├── components # container of all general components │ │ │ └── utils # container of all general functions │ │ ├── views # container of all business pages │ │ │ ├── alarm │ │ │ └── ...... │ │ ├── app-routing.module.ts │ │ ├── app.component.css │ │ ├── app.component.less │ │ ├── app.component.html │ │ ├── app.component.ts │ ├── assets │ │ ├── json # container of mock data assets │ │ ├── i18n # container of internationalization assets │ │ └── images │ ├── environments │ ├── favicon.ico │ ├── index.html │ ├── style.css │ ├── style.less │ ├── my-theme.css │ ├── my-theme.less │ ├── main.ts │ ├── polyfill.ts │ ├── test.ts │ ├── tsconfig.app.json │ ├── tsconfig.spec.json │ ├── typing.d.ts ├── .angular-cli.json ├── CHANGELOG.md # recorder of all the important changes ├── karma.conf.js ├── localproxy.conf.json # config for mock server proxy ├── proxy.conf.json # config for server proxy ├── tsconfig.json ├── package.json └── README.md |
To keep the original function running, this stage of optimization remains the old mock scheme, so the /assets/json folder is still used to hold all mock data. Under this structure, if developers want to develop locally, they have to modify all the routes in service.ts to include local json file. For example, if the developer wants to develop Home module locally, he has to change the online code to local code.
| Code Block | ||||
|---|---|---|---|---|
| ||||
//home.service.ts ——————local codes
…………
baseUrl = "./assets/json/"
url = {
home_serviceData: this.baseUrl + "/home_serviceData.json",
home_alarmData: this.baseUrl + "/home_alarmData.json",
home_alarmChartData: this.baseUrl + "/home_alarmChartData.json",
home_servicebarData:this.baseUrl + "/home_servicebar.json",
sourceNames: this.baseUrl + "/SourceName.json",
listSortMasters:this.baseUrl+"/listSortMsters.json",
}
//home.service.ts ——————online codes
…………
url = {
home_serviceData: this.baseUrl + "/uui-lcm/serviceNumByCustomer",
home_alarmData: this.baseUrl + "/alarm/statusCount",
home_alarmChartData: this.baseUrl + "/alarm/diagram",
home_servicebarnsData: this.baseUrl + "/uui-lcm/ns-packages",
sourceNames: this.baseUrl + "/alarm/getSourceNames",
listSortMasters: this.baseUrl + "/listSortMasters",
} |
And once he has finished the local development and been ready to push the code online, he has to revert previous changes in service.ts.
To avoid mistakes, we used to maintain two set of code, the local one and the online one. The local code is used for local development and the online code is used for online code contribution. Every time the developer finishes local coding, he has to compare the two set of codes, modify the online code and push it online.
2. Optimization Reason
Based on the previous chapter, the optimization reasons are obvious. The current development method makes the developers maintain two set of codes which brings big problems.
Firstly, the developers have to copy and paste codes repetitively which slows down the development speed, as well as increases the rate of code error caused by copy mistakes and omission. Secondly, each time the developers change mock data, they have to change the service module which is the core part of the project. This method increases the risk of fatal mistakes. Thirdly, it is not a graceful way that ‘unrelated’ codes take part in the mock ‘codes’.
For these reasons, building a new mock data scheme is undoubtedly necessary.
3. Optimization Goals
To solve the problems of current mock data scheme, the new one has to achieve the following goals:
Build the ‘one command start’ system to start the project in mock environment and abandon the previous method that uses two set of codes to mock data
Separate the business codes and the mock data codes to make the mock data module an independent one, which can avoid the negative effects of the project core service
Build a real ‘frontend-backend separation’ system. Meanwhile, this new system should be available for the current mock data files as well as be adapted to the requirement of quick development under the condition of server data not ready.
4. Chosen Tools
json-server+faker.jsjson-server is used for build local server and faker.js is used for generating mock data. Click the following links to see more:
json-server:
https://github.com/typicode/json-server
faker.js:
https://github.com/marak/Faker.js/
https://www.npmjs.com/package/faker
5. Optimization Difficulties
With the help of the tools we choose, most request can be mocked. However, some specific condition of the the project cause some difficulties to the building of this scheme:
Some original API paths consist of variable which makes it impossible to be mocked by the local data files
The RESTful standard allows one single API path to hold several different request methods: POST, GET, DELETE, PUT... json-server provides methods to simulate all these different methods but developers have to obey some specific format when sending request, but it can’t be accepted since the original API format doesn’t obey this format.
6. Solutions
For solving the problems above, we use the following methods:
Use the ‘rewrite’ middleware to rewrite specific API paths
Create the ‘routes.js’ file to list the API paths which need to be rewrote
Use the interface interception system to transfer all kinds of request to GET method
7. Optimization Ideas
Image 1
Let’s explain the image above briefly. To achieve the optimization goals and solve the hard problems, we follow the five steps shown in image 1:
Configure package.json, install json-server, set startup command and configure proxy to switch the local server to mock server by proxy
Create server.js to save mock server configuration
Create faker.js to support synchronous development of old and new modules
Configure routes.js to support forwarding data interface and variable request paths
Intercept request, rewrite the paths of the ‘non-GET’ methods and change them to ‘GET’ method
8. Specific Operations
After all the preparation, we can finally start our work step by step.
We separate the operations into two parts: the basic operations and the customized configuration.
8.1 Basic Operations
8.1.1 Configuration of package.json
Since json-server provides basic service to the mock server, let’s setup first.
| Code Block | ||||
|---|---|---|---|---|
| ||||
npm install -g json-server |
And then, we can use json-server in the project. We add the startup command of json-server in package.json which consists of the command that switches the local server to mock server and startup mock server.
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
"mockproxy": "ng serve --proxy-config localproxy.conf.json”, // switch local server to mock server
"mockconfig": "node ./src/app/mock/server.js --port 3002”, // start local mock server
"mock": "npm run mockconfig | npm run mockproxy", |
By those configuration, we have achieved the goal of ‘one command startup’. Later, we should write specific commands to json-server to tell it what to do exactly
8.1.2 Creating server.js
Firstly, we have to create a ‘mock’ folder in the path /app to contain all the configurable files related to mock server, which achieves the goal of separating the business codes and mock data codes. Now, we can do whatever we need in this folder to operate the mock server without any side effects to the business.
Secondly, we create server.js. It is such an important configuration file while json-server is running. We follow the flow image below to write this file:
Image 2
In according to this thought, we write the file as follow:
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
const jsonServer = require('json-server');
const server = jsonServer.create();
const middlewares = jsonServer.defaults();
const customersRouters = require('./routes');
const baseUrl = "/usecaseui-server/v1”;
// Set default middlewares
server.use(middlewares);
// Get mock data
const fs = require('fs');
const path = require('path');
let localJsonDb = {}; //import mock datas
const mockFolder = './src/app/mock/json'; //mock json path folder
const filePath = path.resolve(mockFolder);
fileDisplay(filePath);
function fileDisplay(filePath) {
let fileList = [];
// Return filelist on based of filePath
const files = fs.readdirSync(filePath);
files.forEach((filename) => {
// Get filename's absolute path
let filedir = path.join(filePath, filename);
// Get the file information according to the file path and return an fs.Stats object
fs.stat(filedir, (err, stats) => {
if (err) {
console.warn('Get files failed......');
} else {
let isFile = stats.isFile(); // files
let isDir = stats.isDirectory(); //files folder
if (isFile) {
fileList.push(path.basename(filedir, '.json'));
fileList.forEach(item => {
localJsonDb[item] = getjsonContent(item);
})
}
if (isDir) {
fileDisplay(filedir);
}
Object.keys(fakeoriginalData).map(item => {
localJsonDb[item] = fakeoriginalData[item];
})
}
})
})
setTimeout(() => {
runServer(localJsonDb);
}, 100)
}
function getjsonContent(path) {
let newpath = `./src/app/mock/json/${path}.json`;
let result = JSON.parse(fs.readFileSync(newpath));
return result;
}
function serverRewrite() {
server.use(jsonServer.rewriter(customersRouters))
}
function runServer(db) {
server.use(jsonServer.router(db));
}
server.listen(3002, () => {
console.log('Mock Server is successfully running on port 3002!')
});
|
Let’s explain the code briefly.
Firstly, to reduce the cost of modification, this program read the mock data files and use the file names as forwarding interface paths:
| Code Block | ||||
|---|---|---|---|---|
| ||||
………
let fileList = [];
// Return filelist on based of filePath
const files = fs.readdirSync(filePath);
files.forEach((filename) => {
// Get filename's absolute path
let filedir = path.join(filePath, filename);
// Get the file information according to the file path and return an fs.Stats object
fs.stat(filedir, (err, stats) => {
if (err) {
console.warn('Get files failed......');
} else {
let isFile = stats.isFile(); // files
let isDir = stats.isDirectory(); //files folder
if (isFile) {
fileList.push(path.basename(filedir, '.json'));
fileList.forEach(item => {
localJsonDb[item] = getjsonContent(item);
})
}
if (isDir) {
fileDisplay(filedir);
}
Object.keys(fakeoriginalData).map(item => {
localJsonDb[item] = fakeoriginalData[item];
})
}
})
})
………
function getjsonContent(path) {
let newpath = `./src/app/mock/json/${path}.json`;
let result = JSON.parse(fs.readFileSync(newpath));
return result;
}
……… |
Codes above are responsible for changing the file names to forwarding interface paths. The program reads the folders’ content and estimate file status via ‘readdirSync’ and ‘stat’ methods provided by ‘fs’ module in node. And then it pushes the qualified files to ‘fileList’ array and read the files’ content to create object as follow:
| Code Block | ||||
|---|---|---|---|---|
| ||||
{
………
'uui-sotn_getPinterfaceByVpnId': { 'vpn-binding': [ [Object] ] },
'uui-sotn_getPnfInfo': {
'pnf-name': 'pnf1000',
'pnf-id': '79',
'in-maint': true,
'resource-version': '195',
'admin-status': 'up',
'operational-status': 'up',
'relationship-list': { relationship: [Array] }
},
'uui-sotn_getSpecificLogicalLink': {
'link-name': 'nodeId-79-ltpId-4_nodeId-78-ltpId-4',
'in-maint': false,
'link-type': 'some type',
'speed-value': 'some speed',
'resource-version': '13031',
'operational-status': 'up',
'relationship-list': { relationship: [Array] }
},
xuran_test_data: {
'esr-system-info-id': 'xuran',
'service-url': 'http://10.10.10.10:8080/',
'user-name': 'demo',
'password': 'demo123456!',
'system-type': 'ONAP',
'resource-version': '18873'
}
………
} |
The object above uses the names of json files as object keys and uses the contents as object values, which can be accepted by json-server.
This method needs to rename all the original files. The new names should follow the rules that a file name must contain request path names and underlines. E.g. if we want to create a mock data file for the path: /alarm/form/data, we have to name the json file as ‘alarm_form_data.json’. This operation increases the burden of modifying the existed mock data files’ names, however, just one time modification is enough. What’s more, for the new joined developers, they just have to create a json file and name it under the rules of ‘request paths and underlines’ and they can easily get mock data they want. That’s not a bad idea.
Secondly, this program import ‘middleware’ in json-server, so that developers only have to pass the data object into the ‘router’ middleware to implement forwarding data interface.
| Code Block | ||||
|---|---|---|---|---|
| ||||
function runServer(db) {
server.use(jsonServer.router(db));
} |
Thirdly, The imported data object keys are connected by underlines, e.g. ‘alarm_form_data’, but the real paths are connected by slashes, e.g. ‘/alarm/form/data’, so that’s why we have to configure routes.js to deal with it. What’s more, we need another middleware ‘rewriter’ to apply the configuration written in routes.js.
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
const customersRouters = require('./routes’);
………
function serverRewrite() {
server.use(jsonServer.rewriter(customersRouters))
} |
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
…………
module.exports =
{
///////<-------------general interface--------->/////
"/api/*": "/$1",
"/*/*": "/$1_$2",
"/*/*/*": "/$1_$2_$3",
"/*/*/*/*": "/$1_$2_$3_$4",
/////////////////////////////////////////////////////
} |
Since then, a basic local server has been built. When you tap ‘npm run mock’, most of the APIs can be mocked. Hurray!
8.1.3 Creating faker.js
To achieve the goal of separated development of frontend and backend, we need some powerful tools. Mock.js seems a good choice but for json-server, faker.js is always a perfect partner.
We follow the flow bellow to make faker.js work in our program:
Image 3
Let’s install it first.
| Code Block | ||||
|---|---|---|---|---|
| ||||
npm install faker |
Then, let’s create a /mock/mock folder to manage the fake data generated by faker.js. In our project, we create two files in this folder.
The first one is responsible to the function of generating fake data.
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
const fakeData = require('./fakedata.js');
module.exports = {
//Mock json
'customer_info': fakeData.customer,
'alarm_formdata_multiple': fakeData.customer,
'home': fakeData.home,
'language': fakeData.language,
} |
The second one is responsible to the function of creating fake data object.
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
const fakeData = require('./fakedata.js'); module.exports = { //Mock json 'customer_info': fakeData.customer, 'alarm_formdata_multiple': fakeData.customer, 'home': fakeData.home, 'language': fakeData.language, } |
By the operation above, we have to import the fake data object into server.js and merge it with the object generated by local mock data files.
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
const fakeoriginalData = require('./fake/mock.js'); //import datas generated in fakedata.js
………
files.forEach((filename) => {
……………
if (isDir) {
fileDisplay(filedir);
}
Object.keys(fakeoriginalData).map(item => {
localJsonDb[item] = fakeoriginalData[item];
})
}) |
By all these operations above, we have the capacity to support a totally independent frontend development. Hurray! Hurray!
8.2 Customized Configuration
After met the basic requirement, we have to solve the difficulties discussed in Chapter 5.
8.2.1 Customized Request Routes
As we know that json-server can read the request routes automatically, however, if those paths are ‘irregular’,the basic methods will not work. For example, the program can not handle this kind of request:
| Code Block | ||||
|---|---|---|---|---|
| ||||
createServiceType: this.baseUrl + "/uui-lcm/customers/*_*/service-subscriptions/*+*",
getCustomerresourceVersion: this.baseUrl + "/uui-lcm/customers/*_*",
getServiceTypeResourceVersion: this.baseUrl + "/uui-lcm/customers/*_*/service-subscriptions/*+*", |
The *_* and *+* represent variables. The routes above contains several variables which makes it hard for json-server to handle automatically, so that’s why we need routes.js and configure all the specially routes in it:
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
"/uui-lcm/serviceNumByServiceType/:customer": "/CustomersColumn",
"/uui-lcm/customers/:customer": "/getCustomerresourceVersion”,
"/uui-lcm/customers/:customer/service-subscriptions/:id": "/getServiceTypeResourceVersion", |
By configuring, we clarify all the irregular routes. Next, we just need to import the file into server.js and use it as Chapter 8.1.2
8.2.2 Interface Interception
As we have discussed in Chapter 5, in this project, one single API route can be used by several different request methods and all these requests have their specific response data format which is totally different from that provided by json-server. For example:
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
customers: this.baseUrl + "/uui-lcm/customers", /* get */
deleteCustomer: this.baseUrl + "/uui-lcm/customers", /* delete */
createCustomer: this.baseUrl + "/uui-lcm/customers/", /* put */
……………
getAllCustomers() {
return this.http.get<any>(this.url.customers);
}
createCustomer(customer, createParams) {
let url = this.url.createCustomer + customer;
return this.http.put(url, createParams);
}
deleteSelectCustomer(paramsObj) {
let url = this.url.deleteCustomer;
let params = new HttpParams({ fromObject: paramsObj });
return this.http.delete(url, { params });
} |
As the codes shown, ‘/uui-lcm/customers’ is used by PUT, DELETE and GET method. The program must intercept all those routes and mark them with different request methods, otherwise json-server would not return the response data we create and since we ignore the db.json which is to contain the mock data json-server created, there will be error 404. Here’s the codes in server.js:
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
const fs = require('fs');
const path = require('path');
const jsonServer = require('json-server');
const server = jsonServer.create();
const middlewares = jsonServer.defaults();
const customersRouters = require('./routes');
const baseUrl = "/usecaseui-server/v1";
…………
server.post(`${baseUrl}/*`, (req, res, next) => {
const prefix = req.url.replace(baseUrl, "");
req.url = `${baseUrl}/POST${prefix}`;
req.method = 'GET';
next();
})
server.put(`${baseUrl}/*`, (req, res, next) => {
const prefix = req.url.replace(baseUrl, "");
req.url = `${baseUrl}/PUT${prefix}`;
req.method = 'GET';
next();
})
server.delete(`${baseUrl}/*`, (req, res, next) => {
const prefix = req.url.replace(baseUrl, "");
req.url = `${baseUrl}/DELETE${prefix}`;
req.method = 'GET';
next();
})
…………… |
The codes above uses json-server to intercept data interfaces of different request methods and then rewrite the previous request routes, as well as switch all the request methods to GET. Later, we have to change routes.js and specific json files names to make the changes work.
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
"/PUT/uui-lcm/customers/:name/service-subscriptions/:id": "/PUT_uui-lcm_customers_service-subscriptions”,
"/DELETE/uui-lcm/customers/:customer/service-subscriptions/:id": "/DELETE_uui-lcm_customers_service-subscriptions", |
Image 4
The reason why we do these changes is that at the developing period of a project, frontenders only have to be focused on the request parameters and response data. In other words, request methods means little for a frontender in this period so that we transfer all the request methods to GET temporarily to make mock server normally work.
Now, we have achieved all the optimization goals and a mock server can finally run healthily by one command. Hurray! Hurray! Hurray!
9. Final Result
After the optimization, the mock data scheme has been built. This new scheme supports ‘one command startup’ and allows frontenders develop independently when the backenders are not ready. And of course, this scheme perfectly reuses the original mock data files which makes this optimization the least cost.
At last, I’d like to show the final result of our work. You can also see the codes in the Gerrit repo (https://gerrit.onap.org/r/admin/repos/usecase-ui)
The mock folder structure is as follow:
Image 5
The related configuration in package.json is as follow:
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
"scripts": {
"ng": "ng",
"start": "ng serve",
"server": "ng serve --proxy-config proxy.conf.json",
"mockproxy": "ng serve --proxy-config localproxy.conf.json",
"mockconfig": "node ./src/app/mock/server.js --port 3002",
"mock": "npm run mockconfig | npm run mockproxy”,
………
}, |
The full codes of server.js is as follow:
| Code Block | ||||||
|---|---|---|---|---|---|---|
| ||||||
const jsonServer = require('json-server');
const server = jsonServer.create();
const middlewares = jsonServer.defaults();
const customersRouters = require('./routes');
const baseUrl = "/usecaseui-server/v1";
// Set default middlewares (logger, static, cors and no-cache)
server.use(middlewares);
// Get mock data
const fs = require('fs');
const path = require('path');
let localJsonDb = {}; //import mock datas
const fakeoriginalData = require('./fake/mock.js'); //import datas created in fakedata.js
const mockFolder = './src/app/mock/json'; //mock json path folder
const filePath = path.resolve(mockFolder);
fileDisplay(filePath);
function fileDisplay(filePath) {
let fileList = [];
// Return filelist on based of filePath
const files = fs.readdirSync(filePath);
files.forEach((filename) => {
// Get filename's absolute path
let filedir = path.join(filePath, filename);
// Get the file information according to the file path and return an fs.Stats object
fs.stat(filedir, (err, stats) => {
if (err) {
console.warn('Get files failed......');
} else {
let isFile = stats.isFile(); // files
let isDir = stats.isDirectory(); //files folder
if (isFile) {
fileList.push(path.basename(filedir, '.json'));
fileList.forEach(item => {
localJsonDb[item] = getjsonContent(item);
})
}
if (isDir) {
console.warn("=====> DO NOT support mock data in folder");
fileDisplay(filedir);
}
Object.keys(fakeoriginalData).map(item => {
localJsonDb[item] = fakeoriginalData[item];
})
}
})
})
setTimeout(() => {
serverRewrite();
runServer(localJsonDb);
}, 100)
}
function getjsonContent(path) {
let newpath = `./src/app/mock/json/${path}.json`;
let result = JSON.parse(fs.readFileSync(newpath));
return result;
}
//only multi router data needs jsonServer.rewriter
function serverRewrite() {
server.use(jsonServer.rewriter(customersRouters))
}
function runServer(db) {
server.use(jsonServer.router(db));
}
server.post(`${baseUrl}/*`, (req, res, next) => {
const prefix = req.url.replace(baseUrl, "");
req.url = `${baseUrl}/POST${prefix}`;
req.method = 'GET';
next();
})
server.put(`${baseUrl}/*`, (req, res, next) => {
const prefix = req.url.replace(baseUrl, "");
req.url = `${baseUrl}/PUT${prefix}`;
req.method = 'GET';
next();
})
server.delete(`${baseUrl}/*`, (req, res, next) => {
const prefix = req.url.replace(baseUrl, "");
req.url = `${baseUrl}/DELETE${prefix}`;
req.method = 'GET';
next();
})
server.listen(3002, () => {
console.log('Mock Server is successfully running on port 3002!')
}); |
That's all our jobs. Looking forward to suggestion and discussion.