Administering Oracle Linux 7: Part 1 – Service Management

Administering Oracle Linux 7: Part 1 – Service Management

This article is Part 1 of a series that explains how to administer Oracle Linux 7. This article focuses on service management.

Read Part II
Read Part III

Introduction to Services

Oracle Linux 7 has changed its framework for starting a system, administering services, and configuring services. Now, all management is performed by the systemddaemon (the new system and service manager), which takes care of processes and server daemons during the boot process and when the system is running. Additionally, systemd is able to automatically resolve service dependencies, reducing troubleshooting time and reducing the time required to start services.

In previous versions of Oracle Linux, the init process had process identifier (PID) 1, but on Oracle Linux 7 this PID is associated with systemd, as show below:

[root@oel73 ~]# ps aux | head -2

USER       PID %CPU %MEM    VSZ   RSS    TTY      STAT START   TIME COMMAND

root       1   0.4  0.1    52844  6772   ?        Ss   20:25   0:02 /usr/lib/systemd/systemd --switched-root --system --deserialize 23

There are several types of systemd objects, called units, which can be used during service administration, as shown below:

[root@oel73 ~]# systemctl -t help
Available unit types:
service
socket
target
device
mount
automount
snapshot
timer
swap
path
slice
scope

From all the units listed above, two of them (service and socket) are commonly handled during service administration. The following summarizes the unit types:

• service  represents system services (.service extension).
• socket represents an interprocess communication socket (.socket extension).
• target represents a group of systemd units (.target extension).
• device is the representation of a device as a file to support device-based activation (.device extension).
• mount is used by systemd to monitor system mount points (.mount extension).
• automountis used by systemd to monitor file system automount points (.automountextension).
• snapshot is a representation of a system state (.snapshotextension).
• timer represents a systemd timer (.timer extension).
• swap represents a swap file or swap device (.swap extension).
• path is a directory or file representation that is used to delay the start of a service until a determined status happens (.path extension).
• slice represents a group of organized units to manage system processes (.slice extension).
• scope represents an externally created process (.scope extension).

All these unit types are involved with Oracle Linux 7 initialization, so for a better understanding of this mechanism, the following is the step-by-step process:

• The system firmware executes the power-on self-test (POST), and searches for a boot device configured either in the Master Boot Record (MBR) or the Unified Extensible Firmware Interface (UEFI) boot firmware.
•  The boot loader (GRUB2) takes control.
• The boot loader reads its configuration file from either the/etc/grub.d directory or the/etc/default/grub or /boot/grub2/grub.cfg files. The content of the /etc/default/grubfile is shown below:

[root@oel73 ~]# more /etc/default/grub 
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="crashkernel=auto  vconsole.font=latarcyrheb-sun16 rd.lvm.lv=ol/swap rd.lvm.lv=ol/root vconsole.keymap=us rhgb quiet"
GRUB_DISABLE_RECOVERY="true"

• The boot loader loads the kernel and initramfsfrom disk to memory. Afterwards, all kernel modules from initramfs are also executed.
• The boot loader hands control over to the kernel.
• Drivers from initramfs are load by the kernel.
• The /sbin/init file from initramfs is executed.
• The systemd daemon from initramfs runs all target units (.target extension), which mounts the /sysroot directory.
• Control is switched from memory (initramfs) to disk (system root file system).
• systemd executes a default target (/etc/systemd/system/default.target file) and all related units (/etc/systemd/system directory) to resolve dependencies. At the end, we have either a text-based screen or a GUI.  The contents of the /etc/systemd/system/default.target file is shown below:

[root@oel73 ~]# more /etc/systemd/system/default.target
[Unit]
Description=Graphical Interface
Documentation=man:systemd.special(7)
Requires=multi-user.target
After=multi-user.target
Conflicts=rescue.target
Wants=display-manager.service
AllowIsolate=yes
[Install]
Alias=default.target

The systemd target works as a group of systemd units, which must be started to get to a stable end state (very similar to the milestones for Oracle Solaris

The more relevant targets are listed below:

• multi-user.target represents text-based logins with support for multiple users.
• graphical.target represents graphical and text-based logins supporting multiple users. This target includes the previous one.
• emergency.target indicates the system has transited from initramfs to the system root file system mounted as read-only.
• rescue.target indicates the basic system initialization has been completed.

Administering Services

The first changes introduced by Oracle Linux 7 are the commands to stop and reboot the system, as shown below

[root@oel73 ~]# systemctl poweroff
[root@oel73 ~]# systemctl reboot

Both commands have backward compatibility to the old poweroff and reboot commands, respectively, from previous Oracle Linux versions. These old commands are symbolically linked to the new Oracle Linux 7 systemctl commands.

When managing services on Oracle Linux 7, the main operation is to collect the status of all the services. However, before getting the statuses, you need to know all the existing services on the system. Thus, execute the following:

[root@oel73 ~]# systemctl | more
UNIT                                 LOAD   ACTIVE SUB       DESCRIPTION
proc-sys-fs-binfmt_misc.automount    loaded active waiting   Arbitrary Executable File 
Formats File System Automount Point
sys-devices-pci0000:00-0000:00:10.0-host2-target2:0:0-2:0:0:0-block-sda-
sda1.device    loaded active plugged   VMware_Virtual_S
sys-devices-pci0000:00-0000:00:10.0-host2-target2:0:0-2:0:0:0-block-sda-
sda2.device    loaded active plugged   LVM PV JC02RJ-zNYK-71eq-Ed
3t-GygF-WwTp-sZNbHy on /dev/sda2
sys-devices-pci0000:00-0000:00:10.0-host2-target2:0:0-2:0:0:0-block-sda.device    
loaded active plugged   VMware_Virtual_S
sys-devices-pci0000:00-0000:00:10.0-host2-target2:0:1-2:0:1:0-block-sdb-
sdb1.device    loaded active plugged   VMware_Virtual_S

Sometimes it is convenient to list only the service units, as shown below:

[root@oel73 ~]# systemctl list-units --type service –all
UNIT                                 LOAD   ACTIVE SUB       DESCRIPTION
abrt-ccpp.service                    loaded active exited    Install ABRT coredump hook
abrt-oops.service                    loaded active running   ABRT kernel log watcher
abrt-xorg.service                    loaded active running   ABRT Xorg log watcher
abrtd.service                        loaded active running   ABRT Automated Bug Reporting
Tool
accounts-daemon.service              loaded active running   Accounts Service
alsa-state.service                   loaded active running   Manage Sound Card State (restore and store)
atd.service                          loaded active running   Job spooling tools
...

Now that the existing services are known, to get the status of the firewall service, execute the following command

[root@oel73 ~]# systemctl status firewalld.service
firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled)
   Active: active (running) since Sat 2015-03-28 02:14:33 BRT; 1h 28min ago
 Main PID: 604 (firewalld)
   CGroup: /system.slice/firewalld.service
           └─604 /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid
Mar 28 02:14:33 oel73.example.com systemd[1]: Started firewalld - dynamic firewall daemon.

The firewalld.service service unit shows that the firewalld daemon is active and running. Nevertheless, other relevant statuses are possible, such as inactive (the service is not running), enabled (the service will be loaded at boot time), disabled (the service won’t be started at boot time), static (the service cannot be enabled, but it can be started by another unit automatically), and active (the service is waiting for an event).

Another easy way to verify the status of each service is by running the following command:

[root@oel73 ~]# systemctl list-unit-files --type service
UNIT FILE                                   STATE   
abrt-ccpp.service                           enabled 
abrt-oops.service                           enabled 
abrt-pstoreoops.service                     disabled
abrt-vmcore.service                         enabled 
abrt-xorg.service                           enabled 
abrtd.service                               enabled 
accounts-daemon.service                     enabled 
alsa-restore.service                        static  
alsa-state.service                          static  
alsa-store.service                          static  
anaconda-direct.service                     static  
anaconda-noshell.service                    static  
...

To stop a service and to get its status, execute the following:

[root@oel73 ~]# systemctl stop firewalld.service
[root@oel73 ~]# systemctl status firewalld.service
firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled)
   Active: inactive (dead) since Sat 2015-03-28 03:51:00 BRT; 2s ago
  Process: 604 ExecStart=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS (code=exited, status=0/SUCCESS)
 Main PID: 604 (code=exited, status=0/SUCCESS)

To start the firewalld service again, execute the following:

[root@oel73 ~]# systemctl start firewalld.service

To restart and reload a service, run the following commands:

[root@oel73 ~]# systemctl restart firewalld.service
[root@oel73 ~]# systemctl reload firewalld.service

To verify whether a specific service (for example, firewalld) is active at this time and is enabled to start at boot time, run the following commands:

[root@oel73 ~]# systemctl is-active firewalld
Active
[root@oel73 ~]# systemctl is-enabled firewalld
Enabled

In rare cases, it is necessary to prevent a service from starting because of a possible conflict with another service. However, a disabled service could be enabled accidentally by an administrator and the result could be unpredictable, so this case we could “mask” the service. A masked service doesn’t start automatically, but it starts manually if the administrator runs the systemctl start command. Thus, to mask and unmask a service, execute the following:

[root@oel73 ~]# systemctl mask firewalld
ln -s '/dev/null' '/etc/systemd/system/firewalld.service'
[root@oel73 ~]# systemctl unmask firewalld
rm '/etc/systemd/system/firewalld.service'

During maintenance of s system, it is appropriate to list the failed services and this can be accomplished by executing the following:

[root@oel73 ~]# systemctl --failed --type=service
UNIT            LOAD   ACTIVE SUB    DESCRIPTION
network.service loaded failed failed LSB: Bring up/down networking
rhnsd.service   loaded failed failed LSB: Starts the Spacewalk Daemon
rngd.service    loaded failed failed Hardware RNG Entropy Gatherer Daemon
LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.
3 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.

To find any stuck jobs on the system, run the following:

[root@oel73 ~]# systemctl list-jobs
No jobs running.

Almost all services have required dependencies in order to start, so to list the dependencies of a service, execute the following:

[root@oel73 ~]# systemctl list-dependencies firewalld.service
firewalld.service
├─dbus.socket
├─system.slice
└─basic.target
 ├─alsa-restore.service
 ├─alsa-state.service
 │ └─...
 ├─microcode.service
 ├─rhel-autorelabel-mark.service
...

To list the dependencies in reverse order, run the following:

[root@oel73 ~]# systemctl list-dependencies --reverse firewalld.service
firewalld.service
└─basic.target
  ├─abrt-ccpp.service
  ├─abrt-oops.service
  ├─abrt-vmcore.service
  ├─abrt-xorg.service
  ├─abrtd.service
  ├─accounts-daemon.service
  ├─atd.service
  ├─avahi-daemon.service
...

Previously, we learned about the systemd targets (a set of systemd units). It is easy to list all the available targets by running the following command:

[root@oel73 ~]# systemctl list-units --type=target -all
UNIT                   LOAD   ACTIVE   SUB    DESCRIPTION
basic.target         loaded active   active Basic System
cryptsetup.target      loaded active   active Encrypted Volumes
emergency.target       loaded inactive dead   Emergency Mode
final.target           loaded inactive dead   Final Step
getty.target           loaded active   active Login Prompts
graphical.target       loaded active   active Graphical Interface
local-fs-pre.target    loaded active   active Local File Systems (Pre)
local-fs.target        loaded active   active Local File Systems
multi-user.target      loaded active   active Multi-User System
network-online.target  loaded inactive dead   Network is Online
network.target         loaded active   active Network
...

Additionally, to list all the existing targets on disk, execute the following:

[root@oel73 ~]# systemctl list-unit-files --type=target -all
UNIT FILE                 STATE   
anaconda.target           static  
basic.target               static  
bluetooth.target         static  
cryptsetup.target          static  
ctrl-alt-del.target       disabled
default.target             enabled 
dirsrv.target             disabled
emergency.target           static  
final.target               static  
...

Picking a relevant target, such as graphical.target, we can list its dependency on other targets by executing the following

[root@oel73 ~]# systemctl list-dependencies graphical.target | grep target
graphical.target
└─multi-user.target
  ├─basic.target
  │ ├─paths.target
  │ ├─slices.target
  │ ├─sockets.target
  │ ├─sysinit.target
  │ │ ├─cryptsetup.target
  │ │ ├─local-fs.target
  │ │ └─swap.target

  │ └─timers.target
  ├─getty.target
  ├─nfs.target
  └─remote-fs.target

In previous Oracle Linux versions, it was possible to go to specific run level. In Oracle Linux 7, to go to a specific target (the equivalent of a run level) and stop all services that are not related to the target, execute the following:

[root@oel73 ~]# sysctl isolate multi-user.target

A question arises: What’s the default target on Oracle Linux 7? To find the answer, execute the following:

[root@oel73 ~]# systemctl get-default
graphical.target

This configuration is shown as being persistent in the /etc/systemd/system directory, as shown below:

[root@oel73 system]# pwd
/etc/systemd/system
[root@oel73 system]# ls -al default.target
lrwxrwxrwx. 1 root root 36 Sep 10  2014 default.target -> /lib/systemd/system/graphical.target

To change the default target, run the following:

[root@oel73 ~]# systemctl set-default multi-user.target

Recovering from Boot Issues and Other Problems

The most common problem when administering an Oracle Linux 7 system is losing the root password. Use the following procedure to reset it:

• Reboot Oracle Linux 7.
• Interrupt the boot process by pressing any key.
• Edit the line that starts with linux16 and, at end, append the string rd.break, as shown below:

linux16 /vmlinuz-3.8.13-55.1.5.el7uek.x86_64 root=/dev/mapper/ol-root ro crashkernel=auto  vconsole.font=latarcyrheb-sun16  rd.lvm.lv=ol/swap  rd.lvm.lv=ol/root vconsole.keymap=us rhgb quiet LANG=en_US.UTF-8 initrd16 /initramfs-3.8.13-55.1.5.el7uek.x86_64.img rd.break

• Boot by pressing CTRL + x.
• Execute mount –o remount,rw /sysroot.
• Execute chroot /root.
• Execute passwd root.
• Execute exit.
• Execute exit.
• Execute touch /.autorelabel (because of SELinux protection).

Additionally, to select a different target at boot time, it is necessary to include the string systemd.unit at the end of the line starting with linux16, as shown in the procedure below:

• Reboot Oracle Linux 7
• Interrupt the boot process by pressing any key.
• Edit the line that starts with linux16 and, at end, append either rescue.target or emergency.target, as shown below:

linux16 /vmlinuz-3.8.13-55.1.5.el7uek.x86_64 root=/dev/mapper/ol-root ro crashkernel=auto  vconsole.font=latarcyrheb-sun16  rd.lvm.lv=ol/swap  rd.lvm.lv=ol/root vconsole.keymap=us rhgb quiet LANG=en_US.UTF-8 initrd16 /initramfs-3.8.13-55.1.5.el7uek.x86_64.img systemd.unit=rescue.target

linux16 /vmlinuz-3.8.13-55.1.5.el7uek.x86_64 root=/dev/mapper/ol-root ro crashkernel=auto  vconsole.font=latarcyrheb-sun16  rd.lvm.lv=ol/swap  rd.lvm.lv=ol/root vconsole.keymap=us rhgb quiet LANG=en_US.UTF-8 initrd16 /initramfs-3.8.13-55.1.5.el7uek.x86_64.img systemd.unit=emergency.target

Remember that rescue.target waits for sysinit.target to finish before initializing other components from the system, while emergency.target mounts the root file system as read-only. Both options can be used to fix serious problems that prevent Oracle Linux 7 from booting.

• Boot by pressing CTRL + x.
• Execute mount –o remount,rw /sysroot.
• Execute chroot /root.
• Perform the required maintenance. For example, we could repair the boot load by executing grub2-mkconfig > /boot/grub2/grub.cfg.
• Execute exit.
• Execute systemctl reboot.

I hope you enjoyed this article!

See Also

Here are some links to other things I’ve written:

• Alexandre Borges on Twitter
• Alexandre Borges’ personal blog
• “Exploring Installation Options and User Roles in Oracle Solaris 11“
• “Exploring Networking, Services, and the New Image Packaging System in Oracle Solaris 11“
• ZFS Series https://community.oracle.com/docs/DOC-914874