Category: EM12c

Posted in EM12c, EM13c, Oracle

EM 12c, 13c: Enterprise Manager Cloud Control Agent Install Fails at Initialization Step With Error: SSH daemon (sshd) is not running on port 22

Posted on by mardaff

Screen Shot 2018-06-11 at 9.03.06 AM

This error can occur due to one of the following reasons:

1. SSH port 22 is blocked by proxy / firewall on the Target machine
2. SSH is not running on the Target machine
3. SSH daemon is running on non-default port (other than 22)

1. Verify that the SSH port 22 is not blocked from the OMS host using the following command (a blocked port will be indicated, a free port will result in no response):

telnet 22

2. Verify that SSH is running on the target host using

ps -ef | grep ssh
 netstat -anp | grep 22

The Prerequisites for 12c Agent installation using Push Method clearly states that the SSH daemon should be running on the default port (that is, 22) on all the destination hosts.

3. If SSH on the target host is running on a non-default port, then update the SSH_PORT property in $/oui/prov/resources/Paths.properties as below:

SSH_PORT=
  • If you are installing on target A where SSH is running on port 99; then you would need to make the above change. If the next Agent Deployment is on target B where SSH is running on default port 22; then you will need to revert the changes to set SSH_PORT= or SSH_PORT=22 in $/oui/prov/resources/Paths.properties file.
  • You cannot install Agents on target X and target Y at the same time, if they have SSH running on different ports.
Advertisements
Posted in EM12c, OEM, Oracle

Agent has disabled full monitoring of this target. Only target Status is being monitored – EM12c

Posted on by mardaff

I ran into this “exception”: Agent has disabled full monitoring of this target. Only target Status is being monitored”:

12c_error1

After a couple of researches, I’ve found that this is a Bug and therefor I needed to follow Doc ID 2207479.1.

First you need to make sure that you’re patching the right ORACLE_HOME:

export ORACLE_HOME=/u01/app/oracle/product/agent12cR5/core/12.1.0.5.0

[server1 25104978]$ $ORACLE_HOME/OPatch/opatch lsinventory
Oracle Interim Patch Installer version 11.1.0.10.4
Copyright (c) 2017, Oracle Corporation. All rights reserved.


Oracle Home : /u01/app/oracle/product/agent12cR5/core/12.1.0.5.0
Central Inventory : /u01/app/oracle/oraInv
 from : /u01/app/oracle/product/agent12cR5/core/12.1.0.5.0/oraInst.loc
OPatch version : 11.1.0.10.4
OUI version : 11.1.0.13.0
Log file location : /u01/app/oracle/product/agent12cR5/core/12.1.0.5.0/cfgtoollogs/opatch/opatch2017-02-14_13-44-41PM_1.log

Lsinventory Output file location : /u01/app/oracle/product/agent12cR5/core/12.1.0.5.0/cfgtoollogs/opatch/lsinv/lsinventory2017-02-14_13-44-41PM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (1):

EM Platform (Agent) 12.1.0.5.0
There are 1 products installed in this Oracle Home.


Interim patches (3) :

Patch 10203435 : applied on Thu Jun 11 00:41:03 GMT+00:00 2015
Unique Patch ID: 15915936.1
 Created on 7 Feb 2013, 18:06:13 hrs PST8PDT
 Bugs fixed:
 10203435

Patch 17591700 : applied on Thu Jun 11 00:41:01 GMT+00:00 2015
Unique Patch ID: 16995298
 Created on 21 Nov 2013, 11:54:16 hrs PST8PDT
 Bugs fixed:
 16479818, 6895422, 13583799

Patch 17018143 : applied on Thu Jun 11 00:40:56 GMT+00:00 2015
Unique Patch ID: 17273347
 Created on 7 Feb 2014, 21:45:46 hrs UTC
 Bugs fixed:
 17018143

--------------------------------------------------------------------------------

OPatch succeeded.

$ORACLE_HOME/OPatch/opatch napply

And on EM repository, don’t forget to execute the following script:

SYSMAN@EMREP> @clean_event_bug25097559.sql
cleaning up entries for seq id: ############################
clear_stateless_event code completed

PL/SQL procedure successfully completed.

Elapsed: 00:00:00.89

Commit complete.

Elapsed: 00:00:00.01
SYSMAN@EMREP>
Posted in EM12c, Oracle

How to clean EM12c repository after a failed installation or upgrade

Posted on by mardaff

Well, as nothing is perfect, it happened to me too…..I had a “never seen before” failed EM12c upgrade on a test environment (fortunately :)) and this helped me to do it from scratch:

Note: this is documented on Doc ID 1395423.1 (useful for one who has access, for those who don’t ….you can check it below)

If BI Publisher is not associated with Enterprise Manager 12c Cloud Control, use the following procedure.

Login to the repository database as SYS user and execute the below:

1. Drop the repository users:

drop user SYSMAN cascade;
drop user SYSMAN_MDS cascade;
drop user MGMT_VIEW cascade;
drop user SYSMAN_RO cascade;
drop user SYSMAN_OPSS cascade;
drop user SYSMAN_APM cascade; 
drop user SYSMAN_BIP;
drop user CLOUD_SWLIB_USER cascade;
drop user CLOUD_ENGINE_USER cascade;
drop role MGMT_USER;

2. Drop the synonyms:

SQL> spool synonyms.lst
SQL> select 'drop '|| decode(owner,'PUBLIC',owner||' synonym '||synonym_name, ' synonym '||owner||'.'||synonym_name) ||';' from dba_synonyms where table_owner in ('SYSMAN', 'SYSMAN_MDS', 'MGMT_VIEW', 'SYSMAN_BIPLATFORM', 'SYSMAN_APM', 'SYSMAN_OPSS', 'SYSMAN_RO') ;
SQL> Spool off
SQL> @synonyms.lst

3. Drop the tablespace objects:

SQL> spool tablespaces.lst
SQL> select 'drop tablespace '||tablespace_name||' including contents and datafiles;' from dba_tablespaces where tablespace_name like 'MGMT%';
SQL> Spool off
SQL> @tablespaces.lst

4. Drop the registry objects:

delete from SCHEMA_VERSION_REGISTRY where COMP_NAME='Metadata Services';
delete from SCHEMA_VERSION_REGISTRY where COMP_NAME='Authorization Policy Manager';
delete from SCHEMA_VERSION_REGISTRY where COMP_NAME='Oracle Platform Security Services';
commit;

If BI Publisher is associated with Enterprise Manager for running reports, then follow this procedure instead.
1. Run DBCA to create a template (structure only) of the existing repository database.
2. Run DBCA to drop the instance storing the EM repository.
3. Run DBCA to create a new instance, using the template created from the old repository instance, to support a new EM 12c installation.

Posted in EM12c, Upgrade

EM12cR2 upgrade to EM12cR5

Posted on by mardaff

I followed the steps below:

# backup the rule sets
 To make a copy, from the Setup menu, select Incidents, then select Incident Rules.
 On the Incident Rules - All Enterprise Rules page, in the table, select the out-of-box rule set you want to copy.
 Then, from the Actions menu, select Create Like Rule Set. In the Create Like Rule Set page, provide the required details and click Save.
# Back up the OMS (the middleware home and the inventory)
 [oracle@oem bin]$ /u01/app/oracle/product/11.2/OPatch/opatch lsinventory
 Invoking OPatch 11.2.0.1.1
Oracle Interim Patch Installer version 11.2.0.1.1
 Copyright (c) 2009, Oracle Corporation. All rights reserved.
Oracle Home : /u01/app/oracle/product/11.2
 Central Inventory : /u01/app/oracle/oraInventory
 from : /etc/oraInst.loc
 OPatch version : 11.2.0.1.1
 OUI version : 11.2.0.2.0
 OUI location : /u01/app/oracle/product/11.2/oui
 Log file location : /u01/app/oracle/product/11.2/cfgtoollogs/opatch/opatch2015-11-18_10-09-47AM.log
Patch history file: /u01/app/oracle/product/11.2/cfgtoollogs/opatch/opatch_history.txt
Lsinventory Output file location : /u01/app/oracle/product/11.2/cfgtoollogs/opatch/lsinv/lsinventory2015-11-18_10-09-47AM.txt
--------------------------------------------------------------------------------
 Installed Top-level Products (1):
Oracle Database 11g 11.2.0.2.0
 There are 1 products installed in this Oracle Home.
There are no Interim patches installed in this Oracle Home.
--------------------------------------------------------------------------------
OPatch succeeded.
[oracle@oem u01]$ locate oraInst.loc | grep oms
 /u01/app/oracle/product/12.1.0/middleware/oms/oraInst.loc
 [oracle@oem u01]$ cat /u01/app/oracle/product/12.1.0/middleware/oms/oraInst.loc
 inventory_loc=/u01/app/oracle/oraInventory
 inst_group=oinstall
[oracle@oem u01]$ locate oraInst.loc | grep agent
 /u01/app/oracle/product/12.1.0/agent/core/12.1.0.2.0/oraInst.loc
 [oracle@oem u01]$ cat /u01/app/oracle/product/12.1.0/agent/core/12.1.0.2.0/oraInst.loc
 inventory_loc=/u01/app/oracle/oraInventory
 inst_group=oinstall
[oracle@oem OMS_BACKUP]$ tar -cvzf GRID_Inventory.tar.gz /u01/app/oracle/oraInventory

# Middle-ware Home and EM domain – take a snapshot of your OMS(Oracle Management Service) use the syntax bellow:

 /bin/emctl exportconfig oms [-sysman_pwd ]
 [-dir ] Specify directory to store backup file
 [-keep_host] Specify this parameter if the OMS was installed
 using a virtual hostname (using ORACLE_HOSTNAME=)
cd /u01/app/oracle/product/12.1.0/middleware/oms/bin/
 [oracle@oem bin]$ ./emctl exportconfig oms -sysman_pwd -dir /backup/OMS_BACKUP/
 Oracle Enterprise Manager Cloud Control 12c Release 2
 Copyright (c) 1996, 2012 Oracle Corporation. All rights reserved.
 ExportConfig started...
 Machine is Admin Server host. Performing Admin Server backup...
 Exporting emoms properties...
 Exporting secure properties...
Export has determined that the OMS is not fronted
 by an SLB. The local hostname was NOT exported.
 The exported data can be imported on any host but
 resecure of all agents will be required. Please
 see the EM Advanced Configuration Guide for more
 details.
Exporting configuration for pluggable modules...
 Preparing archive file...
 Backup has been written to file: /backup/OMS_BACKUP/opf_ADMIN_20151118_102429.bka
The export file contains sensitive data.
 Please ensure that it is kept secure.
ExportConfig completed successfully!

# Copy the emkey from the existing OMS to the existing Management Repository.

[oracle@oem bin]$ ./emctl config emkey -copy_to_repos -sysman_pwd Oracle Enterprise Manager Cloud Control 12c Release 2
 Copyright (c) 1996, 2012 Oracle Corporation. All rights reserved.
 The EMKey has been copied to the Management Repository. This operation will cause the EMKey to become unsecure.
 After the required operation has been completed, secure the EMKey by running "emctl config emkey -remove_from_repos".
 [oracle@oem bin]$
[oracle@oemv bin]$ ./emctl status emkey
 Oracle Enterprise Manager Cloud Control 12c Release 2
 Copyright (c) 1996, 2012 Oracle Corporation. All rights reserved.
 Enter Enterprise Manager Root (SYSMAN) Password :
 The EMKey is configured properly, but is not secure. Secure the EMKey by running "emctl config emkey -remove_from_repos".
 [oracle@oem bin]$

# Preserv changes to the memory

[oracle@oem bin]$ ./emctl get property -name 'JAVA_EM_MEM_ARGS'
 Oracle Enterprise Manager Cloud Control 12c Release 2
 Copyright (c) 1996, 2012 Oracle Corporation. All rights reserved.
 SYSMAN password:
 Value for property JAVA_EM_MEM_ARGS for oms All Management Servers is null
 [oracle@oemv bin]$
$/bin/emctl set property -name 'JAVA_EM_MEM_ARGS' -value '-Xms256m -Xmx1740m'

# stop the EM12c demons:

./$/bin/emctl extended oms jvmd stop -all
 $/bin/emctl extended oms adp stop –all
 ./emctl stop oms -all -force
 ./emctl stop agent

# backup the repository using RMAN

shutdown immediate
 startup mount
RMAN> run{
 allocate channel c1 device type disk format '/backup/OMS_BACKUP/%U';
 backup as compressed backupset database PLUS ARCHIVELOG;
 backup current controlfile;
 }
 .........
shutdown immediate
 lsnrctl stop

# Because the DB version is 11.2.0.2 we need to deploy the necessary patches:
p9748749_112020_Linux-x86-64.zip
p11061801_112020_Generic.zip

unzip p9748749_112020_Linux-x86-64.zip
 unzip p11061801_112020_Generic.zip
cd 9748749
 $ORACLE_HOME/OPatch/opatch apply
 cd 11061801
 $ORACLE_HOME/OPatch/opatch apply
sqlplus / as sysdba
 startup

#- Upgrade the OMS

cd EM12cr5
 ./runInstaller

Deselect the checkbox and click Next:

1

2

Click “Yes” and move forward

Select “Skip” and click on Next

3

4

I ignored the warning in my case since I have both packages installed already. Choose “Ignore” and click Next.

5

As we do an upgrade choose as in the image above and click Next.

6.1

Fill in the Middleware Home Location. Be aware that it has to be a new location and with ~15GB free.

7.1

Fill in the necessary password and confirm that you have backed up the repository and click Next.

8

Click OK and move on.

For the following 2 screens click OK and Yes respectively.

109

Below you can see all the components that will be upgraded:

11

Click Next.

Below you can select additional plugins to be deployed:

12

Click Next when you’re done.

This time I’m doing the installation of the new home on an NFS volume:

13.2

 

In the message below is warning me and it’s giving me an advice about how to proceed after the installation has completed:

14

Click OK and move on.

15.1

Click Next to proceed with the deployment:

16

The last thing for the OMS upgrade:

17

Now running the allroot.sh script

18.1

And that was it:

19.1