Category: Linux

I am creating a server (Centos 8) to develop a system that will use an application using REACT as a front end to show data from different sensors. I will use NGINX as a webserver and Node-red as a back end. In this first post I describe how to install NGINX.

Install nginx package using the yum command on CentOS 8:

sudo yum update
sudo yum install nginx

Update firewall settings and open TCP port 80 and 443. Run:

sudo firewall-cmd –permanent –zone=public –add-service=https –add-service=http
sudo firewall-cmd –reload

Enable nginx service by running systemctl command so that it starts at server boot time:

sudo systemctl enable nginx

start the service, run:

sudo systemctl start nginx

Commands to start/stop/restart nginx server

Run command as per your needs.

sudo systemctl start nginx  <– start the server ##
sudo systemctl stop nginx  <– stop the server ##
sudo systemctl restart nginx  <– restart the server ##
sudo systemctl reload nginx  <– reload the server ##
sudo systemctl status nginx  <– get status of the server ##

You must open and enable port 80 and 443 using the firewall-cmd command:

sudo firewall-cmd –permanent –zone=public –add-service=http –add-service=https
sudo firewall-cmd –reload
sudo firewall-cmd –list-services –zone=public

Now nginx is installed. Bellow are some important directories (defaults) i

Config directory – /etc/nginx/
Maibn global config file – /etc/nginx/nginx.conf
TCP ports opened by Nginx – 80 (HTTP), 443 (HTTPS)
Default web document root directory – /usr/share/nginx/html
Access log file – /var/log/nginx/access.log
Error log file – /var/log/nginx/error.log

IoT Linux web

Command line tools are always welcome.  Today i need to setup my MAC to work with IBM Cloudant

Install curl on your MAC if you haven’t done it yet.

1 – You need to setup the Cloudant DB using your IBM Cloud account.

2 – Go to service credentials and get your username, password and hostname

Sample:
"username": "xxxx-xxx-xxx-bluemix",
"password": "12345678901234567890",
"host": "xxxx-xxx-xxx-bluemix.cloudant.com"

3 – Test the connection

curl –v –u xxxx-xxx-xxx-bluemix 'https://xxxx-xxx-xxx-bluemix.cloudant.com'

The command will prompt for the password: 12345678901234567890

You will see a lot off lines but pay attention on the line Authorization: Basic. This will be used to setup acurl (authorized curl) to avoid type the password every time you use curl.

* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Server auth using Basic with user ‘xxxx-xxx-xxx-bluemix’
* Using Stream ID: 1 (easy handle 0x7fabd0805800)
> GET / HTTP/2
> Host: xxxx-xxx-xxx-bluemix.cloudant.com
> Authorization: Basic asddadasdDASDdaDadaDSERQERQRQERQEDSGSFDGSFDG==

4 – Edit the .bash_profile or .bashrc file in any text editor. For example, from the command line, issue the following command:
open –e .bash_profile

5 – Add the following line to the file, then save the file.
alias acurl="curl -s --proto '=https' -g -H 'Authorization: Basic asddadasdDASDdaDadaDSERQERQRQERQEDSGSFDGSFDG=='"

6 – On the terminal run : source .bash_profile

7 – Issue a command using the new alias. For example, issue a command to view your account information.
acurl -X GET 'https://xxxx-xxx-xxx-bluemix.cloudant.com'

TIP: The hostname is not friendly you can edit your hosts file and create an alias:

On MAC terminal type: sudo vi /etc/hosts

ping your hostname to get the ip address

add a line like this one bellow:

<ip address>  xxxx-xxx-xxx -bluemix.cloudant.com  keniodb.cloudant.com

Save the hosts file

test the connection

acurl -X GET 'https://keniodb.cloudant.com

 

Linux MAC

To install atop on Centos you need to enable EPEL Repository

## RHEL/CentOS 7 64-Bit ##
# wget http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-9.noarch.rpm
# rpm -ivh epel-release-7-9.noarch.rpm

Use yum install atop to install on your server.

More information on this link http://www.tecmint.com/how-to-install-atop-to-monitor-logging-activity-of-linux-system-processes/

Linux

Yesterday i start to setup some new servers and need to install several libraries on Centos 7

Yum is the first option and when i start a command like yum install <anylib> i got several erros and yum was locked.

See this short video and solve the problem.

Linux

Two days ago my NAS WD My Book Live  died. I think because an energy problem.  I bought an USB to SATA adapter and follow the instructions bellow.

The steps worked for a My Book Live 3TB. You need to open and disassembly and extract the HD.  If data partition is not ok, you will loose all data.

1) Download the virgin disk image (02.41.05-034) (203MB) from

https://mega.co.nz/#!FZkASQ6T!P3DMWzaOn62jSrG4SECtIDeJeLYEdmz6w6kR9it8U68599

2) Extract with 7-zip and you should have a file called mybook3tb.img
http://www.7-zip.org/download.html208

3) Format an 8GB+ thumbdrive or external usb hard drive to NTFS and copy mybook3tb.img to it

4) Download SystemRescueCd and burn it to a CD
http://www.sysresccd.org/Download658

5) Shut down your computer and remove the cable from your computers hard drive so we don’t erase it by mistake  ### Verify your discs on gparted before proceed and write down the names of /dev/sd…  ####

6) Hook up your MyBookLive to a SATA port on your motherboard or buy an eSATA to SATA cable or dock and hook up the drive that way. You can try a USB to SATA cable but success rates vary

7) Turn on your computer and tell it to boot from the CD drive

8) When the system rescue cd menu comes up, select to “directly start the graphical environment”

9) Open gparted and check to see where your MyBookLive shows under. In my case it shows up under /dev/sda
The drive should contain several EXT3 partitions and a rather large EXT4 partition

If your drive shows up under sdb, sdc, sdd etc.  just substitute the proper name for the drive when getting to
dd if=/mnt/usb/mybook3tb.img of=/dev/sda bs=1M (DON’T ENTER THIS COMMAND YET)

Now in gparted check to see where your thumbdrive shows up. In my case it shows up under /dev/sdb1
If it shows up under sdc1, sdd1, sde1, etc.  just substitute the proper name for the drive when getting to
mount -t ntfs /dev/sdb1 /mnt/usb (DONT ENTER THIS COMMAND YET)

10) In terminal type the commands below (Make sure you change sdb1 (For the thumbdrive or usb hard drive) and sda (for the MBL hard drive) to whatever they show up as in GPARTED)

mkdir /mnt/usb
mount -t ntfs /dev/sdb1 /mnt/usb
dd if=/mnt/usb/mybook3tb.img of=/dev/sda bs=1M

11) Look for any errors in the terminal window (output should look similar to below)
4420+0 records in
4420+0 records out

12) Logout of the SystemRescue CD and shut down the computer

13) Hook up the pcb to the MyBookLive hard drive and screw it down with 2 screws for now and then hook up the ethernet and power cord and try booting the drive again without putting it back in the case

14) Now from the dashboard do a Quick Factory restore to format the data partition

15) When the quick factory restore is complete power down the MyBookLive

16) Put the drive back together as you are done!

I copied from the link https://community.wd.com/t/guide-debrick-mybooklive-v2-0/58546

Linux

 

linux

 

Linux will turn 25 years old on August 25, the day Linus Torvalds sent out his fateful message asking for help with a new operating system. “I’m doing a (free) operating system (just a hobby, won’t be big and professional like gnu) for 386(486) AT clones. This has been brewing since april, and is starting to get ready. I’d like any feedback on things people like/dislike in minix, as my OS resembles it somewhat (same physical layout of the file-system (due to practical reasons) among other things),” he wrote in the comp.os.minix message board. And the rest, as they say, is history

Linux

My MacBook died yesterday. I am waiting the repair.
Now working with my Ubuntu laptop. Some applications must be installed.
Atom was very easy.

Open a terminal window and run the following:

sudo add-apt-repository ppa:webupd8team/atom
sudo apt-get update
sudo apt-get install atom

Linux

When you setup a lab machine you will need this 3 command lines to disable iptables (firewall) on linux.

# service iptables save
# service iptables stop
# chkconfig iptables off

Linux

I was researching Linux and its development. I found the link to the video Revolution OS that shows how beautiful is the story behind the movement of free software and open source. They changed the world.

Linux

In this post i will describe the steps i follow to backup my IBM Connections 5.0.   The TSM Server and Connections Serve are using Linux.

 Install TSM Client on the IBM Connections Server.

Install gskcrypt and gskssl. Normally, these libraries can be found in an archive with the TSM Client.

#rpm -ivh gskcrypt64-8.0.x.x.linux.x86_64.rpm
#rpm -ivh gskssl64-8.0.x.x.linux.x86_64.rpm
# yum localinstall TIVsm-API64.x86_64.rpm
# yum localinstall TIVsm-BA.x86_64.rpm

TSM Client Configuration.

Add the TSM client service to autostart :

# chkconfig --add dsmcad
# chkconfig dsmcad on

Create the configuration files dsm.sys and dsm.opt.

Copy the files from samples /opt/tivoli/tsm/client/ba/bin/

# cp dsm.opt.smp dsm.opt
# cp dsm.sys.smp dsm.sys

I create some Exclude rules to not backup log files and temporary folders.
All other files of the Connections will be included on the baackup.

The content of my dsm.sys :

Obs: Server name and ip address must match your environment

SErvername  <your tsm server name>
   COMMMethod         TCPip
   TCPPort            1500
   Exclude /.../*.log
   Exclude /.../log/.../*
   Exclude /.../tmp/.../*
   Exclude /.../wstemp/.../*
   Exclude /.../javacore*
   Exclude /.../logs/.../*
   SCHEDLOGRETENTION 7 D
   MAXCMDRETRIES 4
   COMPRESSION YES
   MANAGEDSERVICES WEBCLIENT SCHEDULE
   QUERYSCHEDPERIOD 1
   COMMMethod         TCPip
   TCPPort            1500
   TCPServeraddress   <your tsm server ip address>
   PASSWORDACCESS GENERATE
   NODename Connections

Content of dsm.opt:

SErvername <your tsm server name>

TSM  Policy Domain

Tivoli Storage Manager Policy Management are the important set of rules or instructions to TSM server & database for managing client backups in the server storage. Policies are rules that you set at the IBM Tivoli Storage Manager server to help you manage client data. Policies control how and when client data is stored, how long to be stored and when to expire.
For example:

How and when files are backed up and archived to server storage
How space-managed files are migrated to server storage

The number of copies of a file and the length of time copies are kept in server storage

I created a FILEDOMAIN policy for filesystem backup on my TSM Server with the following commands on TSM console:

define domain FILESDOMAIN
       define policyset FILESDOMAIN FILESPOLICY
       define mgmtclass FILESDOMAIN FILESPOLICY FILESMGMTCLASS
       assign defmgmtclass FILESDOMAIN FILESPOLICY FILESMGMTCLASS
       define copygroup FILESDOMAIN FILESPOLICY FILESMGMTCLASS type=backup dest=backuppool VEREXISTS=5 VERDEL=1 RETEXTRA=3 RETONLY=5
       define copygroup FILESDOMAIN FILESPOLICY FILESMGMTCLASS type=archive dest=archivepool RETVER=8
       validate policyset FILESDOMAIN FILESPOLICY
       activate policyset FILESDOMAIN FILESPOLICY

File retention is key for this configuration.  See more information about this topic at: https://www-304.ibm.com/support/docview.wss?uid=swg21224145

Register the TSM Client

Perform the following steps in the TSM Server’s console, if you haven’t registered the new node yet.

register node CONNECTIONS <password> domain=FILESDOMAIN

Create a client schedule and associate the node

TSM Central scheduling helps automate backup, archive, automatic client updates, and other processes. You can schedule IBM Tivoli Storage Manager operations by using an external scheduler or the built-in Tivoli Storage Manager scheduler. You can configure scheduling through Administrative command line or through TSM Operations Center (GUI).

define schedule FILESDOMAIN WEEKDAY_INCREMENTAL action=Incremental options=-subdir=yes objects='"/opt/ibm/*" "/var/ibm/InstallationManager/*"' DAYofweek=WEEKDay startdate=today starttime=02:30 DURation=3 DURUnits=Minutes

Associate the client with the scheduler

define assoc FILESDOMAIN  WEEKDAY_INCREMENTAL CONNECTIONS

 

Verify the setup

Execute the commands bellow to finish the client configuration and verify the session with server and client schedule:

# dsmc query sessionThe output will be like this

IBM Tivoli Storage Manager
Command Line Backup-Archive Client Interface
  Client Version 7, Release 1, Level 2.0
  Client date/time: 01/11/2016 11:07:26
(c) Copyright by IBM Corporation and other(s) 1990, 2015. All Rights Reserved.Node Name: CONNECTIONS
Session established with server IMPLET: Linux/x86_64
  Server Version 7, Release 1, Level 1.100
  Server date/time: 01/11/2016 11:09:15  Last access: 01/11/2016 10:27:50TSM Server Connection InformationHome Server Name........: <tsm server name>
Server Type.............: Linux/x86_64
Archive Retain Protect..: "No"
Server Version..........: Ver. 7, Rel. 1, Lev. 1.100
Last Access Date........: 01/11/2016 10:27:50
Delete Backup Files.....: "Yes"
Delete Archive Files....: "Yes"
Deduplication...........: "Server Only"Node Name...............: CONNECTIONS
User Name...............: root# dsmc query schedIBM Tivoli Storage Manager
Command Line Backup-Archive Client Interface
  Client Version 7, Release 1, Level 2.0
  Client date/time: 01/11/2016 11:39:13
(c) Copyright by IBM Corporation and other(s) 1990, 2015. All Rights Reserved.Node Name: CONNECTIONS
Session established with server <tsm server name>: Linux/x86_64
  Server Version 7, Release 1, Level 1.100
  Server date/time: 01/11/2016 11:41:03  Last access: 01/11/2016 11:27:52    Schedule Name: WEEKDAY_INCREMENTAL
      Description:
   Schedule Style: Classic
           Action: Incremental
          Options: -subdir=yes
          Objects: "/opt/ibm/*" "/var/ibm/InstallationManager/*"
         Priority: 5
   Next Execution: 14 Hours and 49 Minutes
         Duration: 3 Minutes
           Period: 1 Day  
      Day of Week: Weekday
            Month:
     Day of Month:
    Week of Month:
           Expire: Never

Configure TSM to backup DB2

The Tivoli Storage Manager application programming interface (API) provides a library of functions that allow independent software applications and custom-built applications to back up and archive their data to a Tivoli Storage Manager server. The DB2 DBMS also uses the Tivoli Storage Manager API for backup and restore operations.DB2 provides its own backup utility that can be used to back up data at the table space level or the database level. If you set up this utility to use Tivoli Storage Manager as the backup media, DB2 communicates with the Tivoli Storage Manager API for backup and restore operations. Thus, both the Tivoli Storage Manager API client and backup-archive client work together to provide full data protection for your DB2 environment. The API client and the backup-archive client can run simultaneously on the same DB2 server. The Tivoli Storage Manager server considers them separate clients.

Setup TSM API on DB2 machine

Install gskcrypt and gskssland the TSM API

#rpm -ivh gskcrypt64-8.0.x.x.linux.x86_64.rpm
#rpm -ivh gskssl64-8.0.x.x.linux.x86_64.rpm
#yum localinstall TIVsm-API64.x86_64.rpm

Set the environment variablesSet the following DSMI environment variables in either the operating system shell or the
/home/instance_home_dir/sqllib/userprofile file.
DSMI_DIR
DSMI_CONFIG
DSMI_LOG

Important: The DB2 DBMS reads these environment variables during the DB2 instance startup. If you change the variables, you must restart the
DB2 instance for the changes to take effect.

DSMI_DIR
This variable points to the API installation directory. The dsmtca file, the dsm.sys file, and the language files must be in the directory pointed to by the DSMI_DIR environment variable. Setting the DSMI_DIR variable is optional. If it is not specified, the default directory is /usr/tivoli/tsm/client/api/bin64

DSMI_CONFIG
This variable points to the fully qualified path and file name of the Tivoli Storage Manager dsm.opt client options file. This file contains the name of the server to be used.

DSMI_LOG
This variable points to the directory path where the error log file, dsierror.log, is to be created.

export DSMI_CONFIG=/usr/tivoli/tsm/client/api/bin64/dsm.opt
export DSMI_LOG=/home/db2inst1
export DSMI_DIR=/usr/tivoli/tsm/client/api/bin64

Configure the dsm.sys and dsm.opt and put these files on /usr/tivoli/tsm/client/bin64 and change these files permissions to 775.

You can use the sample from step 3 just exclude the Exlude entries and change the node name.

Log off and log in again as an instance user and run the .profile file.

Restart  the db2 instance.

Register a new Policy Domain for DB2

define domain dbdomain
  define policyset dbdomain dbpolicy
define mgmtclass dbdomain dbpolicy dbmgmtclass
assign defmgmtclass dbdomain dbpolicy dbmgmtclass

define copygroup dbdomain dbpolicy dbmgmtclass type=backup dest=backuppool VEREXISTS=1 VERDEL=0 RETEXTRA=0 RETONLY=0
     validate policyset dbdomain dbpolicy
activate policyset dbdomain dbpolicy

Register the DB2 node

On the TSM Console register the new node:

register node ConnectionsDb2 <password> domain=DBDOMAIN

Set the API user

To access the Tivoli Storage Manager server, client users (called nodes) must have a password to access the server. The DB2 dsmapipw program uses the Tivoli Storage Manager API to create the encrypted password file. The DB2 application includes the dsmapipw utility, which is installed in the /home/instance_home_dir/sqllib/adsm directory.Log in as root user to run the dsmapipw utility. Before you run dsmapipw, you must set the DSMI environment variables similar to that on the DB2 instance.

     # ./dsmapipw

Db2 online backup

On the TSM Server set the following to avoid errors. For detais see this TN http://www-01.ibm.com/support/docview.wss?uid=swg21686874

setopt BackupInitiationRoot no

Update the dabases configuration:

db2 update db cfg using LOGARCHMETH1 TSM (for all IBM Connections databases)

Schedule DB2 backup

Schedule the backup for DB2 databases are different from the filesystem. There are some paths to do the same thing.
I choose a script with cron.Create a file named backup.ksh with the following linesdb2 backup db fnos online use TSM

db2 backup db opnact online use TSM
db2 backup db cognos online use TSM
db2 backup db projexec online use TSM
db2 backup db metrics online use TSM
db2 backup db sncomm online use TSM
db2 backup db wikis online use TSM
db2 backup db forum online use TSM
db2 backup db fngcd online use TSM
db2 backup db homepage online use TSM
db2 backup db dogear online use TSM
db2 backup db peopledb online use TSM
db2 backup db mobile online use TSM
db2 backup db files online use TSM
db2 backup db blogs online use TSM

Add the following entry on the crontab of the root user using crontab -e

00 3 * * 1-6 su - db2inst1 -c "/home/db2inst1/backup.ksh >> /home/db2inst1/fullbackup.log 2>&1"

Managing old DB2 backups

Purge old DB2 backups is not managed by the TSM server.
The backups ara managed by the DB2.  I use a new cron entry and a new scriptCreate a file called deldb2backup.ksh with the following lines:

db2adutl delete FULL older than 4 days  db FILES
db2adutl delete FULL older than 4 days  db OPNACT
db2adutl delete FULL older than 4 days  db COGNOS
db2adutl delete FULL older than 4 days  db PROJEXEC
db2adutl delete FULL older than 4 days  db METRICS
db2adutl delete FULL older than 4 days  db FNOS
db2adutl delete FULL older than 4 days  db SNCOMM
db2adutl delete FULL older than 4 days  db WIKIS
db2adutl delete FULL older than 4 days  db FORUM
db2adutl delete FULL older than 4 days  db FNGCD
db2adutl delete FULL older than 4 days  db HOMEPAGE
db2adutl delete FULL older than 4 days  db DOGEAR
db2adutl delete FULL older than 4 days  db PEOPLEDB
db2adutl delete FULL older than 4 days  db MOBILE
db2adutl delete FULL older than 4 days  db BLOG
Add the following line to root crontab

00 14 * * 6  su - db2inst1 -c "/home/db2inst1/delbackup.ksh >> /home/db2inst1/delbackup.log 2>&1"

 

 

Connections Linux

Today i receive a disk full alarm from my portal server. As usual i delete some logs and files left behind, but for this time this does not solve the issue.

Search for big files or what folder is bigger on linux is hard using du. Googling i found a nice toll and what i found was a big surprise.

I setup NCDU (NCurses Disk Usage) is a curses-based version of the well-known “du”, and provides a fast way to see what directories are using your disk space.

The setup is easy:

1 - yum install ncurses-devel ncurses

2 – download ncdu

3 – untar the file

4- ./configure –prefix=/usr

5 – make && make install

To run just type ncdu at / and the tool will scan your entire server.

Navigating on the toll i found a big folder inside of the connections portlets called “logs” full of files 5GB. The path on my server is :

/opt/IBM/WebSphere/wp_profile/installedApps/portalCell/PA_icWEFPtlts.ear/snor.pf.portlets.war/WEB-INF/logs

I check my server and no tarces are on.

Why these logs are there? I don’t know yet.  I just clean my server.

NCDU did the trick !  Thanks.

Linux portal

Yesterday i was installing a TSM Server for our company. The db2 installer complain about the /tmp size.

The installer need 2,5GB and my /tmp has only 2 GB.

I create a “disk” using dd command with 4GB on a partition with lots of space.

dd if=/dev/zero of=/usr/tmpDSK bs=1024k count=4096
mkfs -t ext3 /usr/tmpDSK
file /usr/tmpDSK
umount /tmp

mount -o loop,exec,nosuid,rw /usr/tmpDSK /tmp

After the db2 installation i umount the /tmp and mount it back using fstab -a

Linux Uncategorized