Author: jude santos

Jude is an electronics engineer and is a self-taught programmer with more than 10 years of experience in building web, desktop and mobile applications. He is currently working on the guttom project.

6 Comments

How to launch the browser based Google Map Directions app instead of the native version.

For Progressive Web Apps (PWA), invoking Google Maps using the maps URL on Android by default, launches the native version of Google Maps, if it is installed on the device. See Maps URL guidelines.

In some situations, we would like to use the browser-based Google Maps, regardless if the native app is installed on a device or not.

In my Ionic 4 / Angular 8 / Capacitor based application, I had a situation where loading Google Driving Directions in the native app would not properly set the source and destination addresses. I also prefer to use the browser-based map as it requires less resources. By changing the URL parameters it is possible to force launch the browser app all the time, by simply replacing the URL below,

https://maps.google.com/maps/dir/?api=1&destination= + destination

with —

https://maps.google.com?saddr=Current+Location&daddr= + destination

I was able to achieve the desired effect.

For a more detailed code implementation. See below:

  async showDrivingDirections(place: any) {
    const destination = encodeURI(place.name);
    if (this.platform.is('android')) {
      if (this.platform.is('mobileweb')) {
        await Browser.open({url: 'https://maps.google.com?saddr=Current+Location&daddr=' + destination});
      } else {
        await Browser.open({url: 'https://maps.google.com/maps/dir/?api=1&destination=' + destination});
      }
    } else {
      window.open('https://maps.google.com/maps/dir/?api=1&destination=' + destination, '_self');
    }
  }

This implementation supports invocation of Google Maps from iOS and Android app with provision to distinguish Progressive Web App (PWA) to achieve uniform behavior for all versions of the apps.

Leave a comment

The Guttom chronicles

Guttom is a food ordering application for the under-served restaurant establishments.

In the next series of articles, I will document the steps, processes and technologies involved in building the front-end and back-end components for guttom.

The screen shots included here will later be incorporated into its own article, expounding the development steps, and the choices of technolog. How each technology choices were made, and the challenges to overcome, which culminated into the Guttom micro services application.

 

Technologies:

Back-end – Java 8, Spring, Spring Boot, Hibernate, JPA, Spring Security OAuth2-JWT, zuul, eureka, spring config, mysql, REST, postman

Front-end – ES6/Typescript, angular5, ionic3, cordova, xcode, android studio, sass/html, karma/jasmine, protractor

Tools/OS: Git, Bitbucket, vim, vscode, eclipse, jenkins, vagrant, docker, virtualbox, ubuntu, Android, iOS, macOS, chrome, Firefox

 

Guttom Apps

 

Guttom Development Environment

 

Guttom Architecture

 

CI/CD with Jenkins

3 Comments

Part 1 – How to integrate ionic’s native filetransfer to OAuth2 JWT in Java’s Spring Boot

Ionic Framework has become formidable in the realm of mobile computing. Ionic allows an enterpreneur/developer to formulate, develop, test, deploy a product idea quickly and introduce that product into the marketplace faster than the competition.

One of the important features of a mobile application is the capability to share photos, images, and files. The Ionic application must have file transfer capabilities. File transfer involves copying, moving, deleting files from and into the mobile device. Ionic provides this capability by tapping into the Apache Cordova native library.

Cordova is a library that sits between ionic and the mobile platform or device operating system (e.g.: ios, android), and provides access to mobile device specific resources – such as the camera, audio, keyboard, and of course, file access.
Cordova provides these resources as a set of APIs in the form of plugins. Ionic needs to install the native plugins in order to have access to the API provided by the mobile device it runs on.

In the case of file transfer the plugin name Is cordova-plugin-file-transfer. More details are provided in the ionic developer docs site (https://ionicframework.com/docs/native/file-transfer/). Cordova file transfer uses http underneath as the messaging protocol.

So where does oauth2 come into play?

Most mobile applications would have some form of security mechanism provided to ensure the integrity of the communications channel between a mobile application and its backend system, or the providers of the resources to the front-end.

In order for the file transfer service to use this communications channel, it has to conform to the security guidelines provided by the communications channel.
In this case we want file-transfers to occur well within the confines of oauth2 security mechanism, to ensure that files being stored in the servers are not compromised; and there is no security loophole in the way these files are being moved into and out of the system.

So how do we incorporate and make file transfers secure using oauth2?

First we need to understand how oauth2 jwt works:

Oauth2

Oauth2 is version 2 of the oauth protocol, it is an authorization framework whereby two parties are guaranteed a secure channel by the authorization server by providing a secure token that the counter party can use to validate the authenticity of the other’s information.

JWT

JWT (JSON Web Tokens) is the token format used by oauth2. A token is a random string generated by the auth. server and is issued to a pre-registered resource user when it is requested at the start of a session or during authentication.

Client Registration

A resource user, or an application client must be registered and recognized by the auth service. The registration process is outside of the definition of oauth2, it only defines the parameters in the request and those to be returned by the auth. server.

Registration is only done once for every application client. The corresponding clientid and client-secret is then used every time the client authenticates to send requests to the resource servers.

ClientId and Secret

A registered resource user, or client is issued a ‘client id’ and a ‘client-secret’. This is then used to authenticate the client at the start of every session, and prior to the auth. service generating the tokens to be used for that session.

Session-less

By the way, oauth is a session-less protocol, this is to contrast with the traditional method whereby 2 entities maintain state after each request, throughout the life of the session.
In oauth, there is no state being maintained, each single request is atomic and is complete by itself. Thus rendering it less vulnerable to common session based security loopholes.

Oauth and session-less communication is also very suitable for micro services based systems, where a resource user will be talking to one or more resource servers at a time.

Tokens

Oauth2 will issue 2 types of tokens every time a jwt token is generated;

Access token is used and added to the http header in the request to the resource server. It has a limited lifetime. The limited lifetime of the token is actually a security mechanism. By constantly issuing new tokens for the lifetime of the session it prevents malicious users from latching on to a token to hijack the session.
Refresh token is used to send a token renewal request to the auth. server when the access token has expired. Thus the session will go unimpeded as long as we are able to send renewal requests and receive a new access token for subsequent client requests.

Java and Spring Security OAuth2 with JWT

You may be wondering why use Java Spring, and not some other framework. I have one micro services project built using java and spring. Java EE has always been the framework of choice when building large scale applications. I always find it faster to implement enterprise services using Java EE. I found it very convenient not having to look far for a security mechanism to incorporate into my application as it is already built in to the framework. It saved me a lot of time adding API security capabilty into my application. Thanks to Spring and it’s enterprise capabilities, you can build fast and focused on business requirements all throughout the implementation stage because the framework already has all the technical infrastructure required to build the system. The development effort was pleasant, and it was all a breeze actually. There should be no room for compromise or experimentation when It comes to implementing large scale systems.

Spring OAuth 2 is an implementation of OAuth2 on top of Spring Security. The authentication mechanism is provided by Spring Security. Illustration below shows the simplified authentication and authorization flow using Spring Security OAuth2 using JWT.

Spring Security OAuth2-JWT authentication & authorization flow diagram

For this topic, I think it will suffice to be able to explain in simple terms how oauth2 jwt works. To dive into more details, there are tons of materials out there that can show the step by step configuration and programming of Spring Security OAuth2 JWT, here’s one link that I highly recommend – http://www.baeldung.com/spring-security-oauth-jwt.

In the next article, we will go through the example steps to incorporate file transfer into the oauth2 jwt. Until then, ciao.

1 Comment

How to set-up a ping service to speed up website page load

So what is the purpose of a pinging service?  You go to the browser and access the website, that’s it. Right? What has pinging a website got to do with reducing the loading time?

One key factor that determines a website’s loading speed is the hosting plan and subscription. Some of my clients complain that their websites are sometimes taking forever to load. Upon further investigation, it turns out that the client is subscribed to a third party hosting service where the server is shared among several websites. It means that the server memory, disk space, and CPU cycles are shared by all sites running on the same server.

In such environments, all websites are competing for resources. A website that often gets site visits (e.g.: SiteA) in a given period are given priority and is cached in server memory. Thus a visitor request is served faster as the resources are already loaded in memory.

In this competing environment, a rarely visited site (e.g.: SiteB) gets pushed out of the cache memory. When a visitor tries to access SiteB, resources for SiteB would first be loaded in memory from the server hard disk. Only after everything is loaded will the site be able to serve the visitor request which, would take at minimum of 3 seconds and oftentimes up to 12 seconds or more, depending on the proximity of the server to the visitor’s location and the size of the page being served.
Any page loading delay beyond 3 seconds is considered too slow.

Loading time increase is directly proportional to the number of sites simultaneously hosted in the shared host server.

How does it impact the website owner. When a website is fairly new and the client is still in the process of marketing and ramping up site visits, it is most crucial that the website load really fast. It doesn’t help the marketing campaign when a prospective visitor navigates to the site and is greeted with a slow loading webpage.

The obvious solution for me is to recommend the upgrade to the client’s hosting service and opt for a private server. That means, the host server will be solely owned by the client’s website, no sharing of memory, disk, and CPU resources. But some clients are on a tight budget and would not, or could not afford to spend more on technical services that they look at as a necessary evil. Believe me, it sometimes is frustrating, but sometimes you I take it as a challenge and try to customize, and at the same time make my clients happy.

My workaround solution to this dilemma is based on the theory that if we can keep a constant flow of simulated site requests at uniform intervals per day, this will keep the host server happy and not unload the site resources in server memory, and thus eliminate the long loading times.

First, we need to create a polling service that can send site requests at a specified interval. But it requires a dedicated server that will be running uninterrupted 24/7 . You may use any low power, low memory computer that can host any of the popular operating systems for this. Just make sure it stays up and running all the time, and also it must have access to the internet.

There are several possible solutions to building a polling service. In my case, I implemented it using Ubuntu Linux bash script to send the actual request using wget, and systemd.

  • wget is a command line executable used to download a webpage.
  • systemd will turn our bash script into a daemon.

A daemon, is simply a long running process, that is never unloaded unless you power down or restart the machine. But once the machine is restarted it will automatically reload and run the polling service for you, so no problemo.

My implementation will be using Ubuntu 16.04 Linux OS, running on an old T1600 Xeon server. The T1600 is also used for several other software development projects I have going, like a jenkins/docker/vagrant build system for a micro services project, a mysql/pgsql db server, a java tomcat platform, a php apache server platform, a virtualbox server host, etc.. I will highlight each of these in future posts.

The following steps are all done using linux terminal bash commands, you may search google for the purpose and usage of each terminal commands as we go.

How to build the bash script:

  1. Create a directory where you want to keep all the necessary files for the ping service.
    The first command will create a directory in the current path named your site-ping. You may use a different name.
    The second command will change our path and bring us inside the your site-ping directory.
user?> mkdir yoursite-ping 

user?> cd yoursite-ping
  1. Create a file named, yoursite-ping.sh. This will be our bash script that will execute the ping request to the website.
user?> touch yoursite-ping.sh
  1. Edit file and add the following commands in the shell script. In this example I will use vim command line editor.

 #!/usr/bin/env bash 
for (( ;  ;)) 
do 
wget -q -O- http://www.yoursite.com > /dev/null
sleep 20m
done

What each line does:

The first line simply instructs the calling command that the following lines will be interpreted by the bash command.

The for do done directive, is a looping instruction. In this implementation we are creating an infinite loop, as described by the (( ; ; )) in the for line.

wget switches -q and -O-  instructs wget to run quietly and send results /dev/null, respectively.

sleep after each loop, we want the request to pause for a given period. In this case we set the duration to 20 minutes. Set the period lower in case you don’t get good results.

  1. Save the file then make it executable. After saving and exiting from vim, chmod the script file.
user?> sudo chmod u+x yoursite-ping.sh
  1. Copy file to a system path. In this case, I use /usr/local/sbin.
 user?> cp yoursite-ping.sh /usr/local/sbin/

How to build the systemd daemon service:

  1. In the same directory, create a file named yoursite-ping.service
user?> touch yoursite-ping.service
  1. Edit yoursite-ping.service using vim, add the following lines.

[Unit] 
Description=yoursite.com Ping Service 
After=network.target 

[Service]
Restart=always
ExecStart=/usr/local/sbin/yoursite.com/yoursite-ping.sh

[Install]
WantedBy=multi-user.target

This is all we need to allow our ping service to load as a daemon and will restart on system reboot or a crash.

The After=network.target line, instructs systemd to load ping service only after the network service is loaded and connected to   the internet.

  1. Save the changes in yoursite-ping.service, copy file to /etc/systemd/system/
 user?> sudo cp yoursite-ping.service /etc/systemd/system/

This will allow systemd to pickup our service configuration on restart and locate our executable script.

  1. Instruct systemd to accept our service and start the service:
user?> sudo systemctl daemon-reload 

user?> sudo systemctl start yoursite-ping.service
  1. Check daemon status. Use either of the 2 commands below:
user?> systemctl list-units --type=service --state=running | grep yoursite-ping

If the previous steps went with no hitch, the above command should show the line

yoursite-ping.service loaded active running yoursite.com Ping Service

or

user?> systemctl status yoursite-ping

Again, if everything went well in the previous steps. It will show the lines;

Loaded: loaded (/etc/systemd/system/yoursite-ping.service; enabled; vendor preset: enabled) 

Active: active (running) since Tue 2015-04-24 15:36:33 UTC; xxh xxmin ago

You’re done! Send comments if you run into issues.

Although I wouldn’t recommend this solution as a permanent fix, it helps to have a way to resolve such issues and adapt as we go, especially on a very tight budget.

See my future posts on website optimization techniques.

4 Comments

Mysql server on vagrant, virtualbox

Vagrant is a popular tool used by development teams to quickly install and deploy one or several clusters of virtualized server environments on which to test and develop software and distributed systems. In the era of micro services, it is extremely important to be able to quickly setup and tear down a test environment that would closely mimic it’s cloud environment once a system is deployed in production.

There are countless possible ways of using vagrant to automate and configure development environments, but for this discussion we will focus on mysql server and ubuntu, and the steps on how to get it up and running quickly. This includes virtualizing and automating the download, setup of ubuntu where mysql server will be running on, as well as the installation of the necessary dependencies, configurations required for mysql server to operate properly.

Ubuntu is a linux based operating system and is one of the most popular among the open source operating systems. We are going to use ubuntu 16.04 in this demonstration.

mysql is also one of the most popular and important open source DB systems today. For this demo, we are going to use mysql 5.7

For vagrant to work, we also need a virtualization provider, for this we are going to use Oracle’s VirtualBox.

We also need a GUI Admin interface to manage and configure mysql databases and its users. For this we are going to use the web based phpmyadmin, as well as the corresponding web tools apache2, and php7.

The platform

Vagrant and virtual box needs to be installed in a host system, for this demo, I am doing it on ubuntu 16.04. You may also do the same steps on Windows, or Mac, only the tools that will be used are different.

Let us begin

On your host system, open a terminal. All succeeding steps will be done on the terminal command line interface, so for those of you who are not familiar with linux commands, you may search google and learn about each of the commands used here as we go along.

1.) Install vagrant


user$> sudo apt-get install virtualbox
user$> sudo apt-get install vagrant

Thats its! Once the download completes both tools should now be available. To check for proper installation of vagrant, type the command


user$> vagrant version

It should return with the following lines


Installed Version: 2.0.3
Latest Version: 2.0.3

2.) Setup vagrant workspace

To use vagrant, create a directory where you want to configure a vagrant work environment. This step is important as vagrant will install a hidden folder name .vagrant as well as file named Vagrantfile. Vagrantfile is where we define the environment to setup and deploy our mysql server.

For this example, I am going to create a directory named mysql-env,


user$> mkdir mysql-env
user$> cd mysql-env

The last command should now take you inside ./mysql-env. Type the command below to initialize the workspace.


user:mysql-env$> vagrant init

3.) Configure Vagrantfile

The file Vagrantfile will be created. Open Vagrantfile using your favorite editor, In my case I will use vim. (To install: sudo apt-get install vim).

The initial content of the file will look like,


Vagrant.configure(2) do |config|
    config.vm.box = “base”
end

config.vm.box specifies the type of operating system that will be used as guest operating system running in virtualbox.
Now change “base” to “xenial64”, this will install ubuntu16.04 the 64bit version. The OS will be downloaded from Vagrant’s cloud OS box site.

Add the additional entries as shown below,


Vagrant.configure(2) do |config|

    config.vm.box = "ubuntu/xenial64"

    config.vm.define "db-server" do |db|
        db.vm.network "public_network", :bridge => "eth0", ip: "192.168.1.237"
        db.vm.network "forwarded_port", guest: 3306, host: 3306
        db.vm.network "forwarded_port", guest: 80, host: 8306
        db.vm.provision "shell", path: “bootstrap.sh"
    end
end

What each line is about,

config.vm.define, allows us to define a named virtual environment. We can define several virtual environments in a single Vagrantfile, as long as we provide a unique name for each. In this particular example, we named our environment to “db-server”.

db.vm.network “public_network”, defines the network interface to use. In this case we are using the host’s dhcp provider (as specified by “public_network”) to assign the ip via a bridge, using the shown IP. Note: the IP value may be changed to suit your network environment. Also make sure the IP is configured in the local network provider/router to lease the IP semi-permanently to this virtual machine as this may change on your next restart of the guest machine, and may have to be reconfigured to a new one.

db.vm.network “forwarded_port”, this will allow us to open a listening port in the host and guest OS. The host OS has the actual connection to the outside world, so it forwards all received packets to the port we specify for the guest OS.

For mysql, we can set 3306 for the guest and host OS.
For the web application phpmyadmin, guest is set to 80, for the host we set to 8306. NOTE: vagrant will complain about port 80 being a well known port and can not be used. Choose any number higher than 1024 and you should be ok.

db.vm.provision “shell”, this allows us to execute a shell script inside the guest OS right after it boots up.
bootstrap.sh, This is where we add additional commands to install and configure the guest OS to run mysql server and phpmyadmin. bootstrap.sh may be renamed to anything you fancy.

4.) Bootstrap.sh.

The contents below should include all we need to configure and install mysql and phpmyadmin.
Change the database name, user, password variables as needed.


#!/usr/bin/env bash

DBHOST=localhost
DBNAME=dbname
DBUSER=dbuser
DBPASSWD=userpass

apt-get update
apt-get install vim curl build-essential python-software-properties git

debconf-set-selections <<< "mysql-server mysql-server/root_password password $DBPASSWD"
debconf-set-selections <<< "mysql-server mysql-server/root_password_again password $DBPASSWD"
debconf-set-selections <<< "phpmyadmin phpmyadmin/dbconfig-install boolean true"
debconf-set-selections <<< "phpmyadmin phpmyadmin/app-password-confirm password $DBPASSWD"
debconf-set-selections <<< "phpmyadmin phpmyadmin/mysql/admin-pass password $DBPASSWD"
debconf-set-selections <<< "phpmyadmin phpmyadmin/mysql/app-pass password $DBPASSWD"
debconf-set-selections <<< "phpmyadmin phpmyadmin/reconfigure-webserver multiselect none"

# install mysql and admin interface

apt-get -y install mysql-server phpmyadmin

mysql -uroot -p$DBPASSWD -e "CREATE DATABASE $DBNAME"
mysql -uroot -p$DBPASSWD -e "grant all privileges on $DBNAME.* to '$DBUSER'@'%' identified by '$DBPASSWD'"

cd /vagrant

# update mysql conf file to allow remote access to the db

sudo sed -i "s/.*bind-address.*/bind-address = 0.0.0.0/" /etc/mysql/mysql.conf.d/mysqld.cnf

sudo service mysql restart

# setup phpmyadmin

apt-get -y install php apache2 libapache2-mod-php php-curl php-gd php-mysql php-gettext a2enmod rewrite

sed -i "s/AllowOverride None/AllowOverride All/g" /etc/apache2/apache2.conf

rm -rf /var/www/html
ln -fs /vagrant/public /var/www/html

sed -i "s/error_reporting = .*/error_reporting = E_ALL/" /etc/php/7.0/apache2/php.ini
sed -i "s/display_errors = .*/display_errors = On/" /etc/php/7.0/apache2/php.ini

service apache2 restart

5.) Boot up db-server

In step 2, We named the virtual machine “db-server”, we use this name to refer to the guest server when sending commands to it.

To start db-server


user:mysql-env$> vagrant up db-server

To check status


user:mysql-env$> vagrant status

To ssh to guest


user:mysql-env$> vagrant ssh db-server

The last command should now bring you inside the db-server. Type cd /vagrant, this will bring you to the base path of the guest machine, which is the same path as your host machine, in mysql-env.

6.) Test connections

Load mysqladmin. Use any browser in the host, or any system in the same network. Login with the username and password you assigned in step-4, bootstrap.sh

On your browser, go to: http://192.168.1.237:8306/phpmyadmin

Connect to mysql server in the host terminal, or any system in the same network. Login with the username and password you assigned in step-4, bootstrap.sh


user$> mysql -u dbuser -p userpass -h 192.168.1.237

That’s it! If all went well, you are on your way to create some more guest services for other purposes, like a spring boot micro service application, or a LAMP/LEMP server, or a Jenkins CI system, the possibilities are endless.

Hope this helps to jumpstart your understanding of vagrant and see for yourself how easy it is to get started. Enjoy and message me if you find any issues with the steps or the scripts.