Thread-1 ERROR main: nmectl.c

Grid Control Agent Startup Fails on AIX Server due to Invalid Timezone Setting at the OS [ID 825933.1]
      Modified 19-NOV-2010     Type PROBLEM     Status PUBLISHED  
 



In this Document
  Symptoms
  Cause
  Solution

Applies to:
Enterprise Manager Grid Control - Version: 10.2.0.1 to 10.2.0.5 - Release: 10.2 to 10.2
IBM AIX on POWER Systems (64-bit)
IBM AIX Based Systems (64-bit)IBM AIX5L Based Systems (32-bit)IBM AIX Based Systems (32-bit)
Symptoms

New installation of the Grid Agent on AIX server fails to start with the below error:
$ emctl start agent
Oracle Enterprise Manager 10g Release 10.2.0.2.0.
Copyright (c) 1996, 2006 Oracle Corporation. All rights reserved.
Starting agent ..... failed.
The agentTZRegion value in /rfes/oracle/OEM10g/agent10g/sysman/config/emd.properties is not in
agreement with what agent thinks it should be.Please verify your environment to make sure that TZ setting has not changed since the last start of the agent.
If you modified the timezone setting in the environment, please stop the agent and exectute 'emctl
resetTZ agent' and also execute the script mgmt_target.set_agent_tzrgn(<agent_name>,
<new_tz_rgn>) to get the value propagated to repository.
Consult the log files in: /rfes/oracle/OEM10g/agent10g/sysman/log

-  Trying to reset the Agent timezone also fails:
$ emctl resetTZ agent
Oracle Enterprise Manager 10g Release 10.2.0.2.0.
Copyright (c) 1996, 2006 Oracle Corporation. All rights reserved.
Updating /rfes/oracle/OEM10g/agent10g/sysman/config/emd.properties...
----- Sun May 3 17:45:51 2009::tzOffset for -05:30 is -330(min), but agent is runnning with
tzOffset -270(min)-----
----- Sun May 3 17:45:51 2009::trying again after waiting for 1 sec to account for daylight
transition -----
----- Sun May 3 17:45:51 2009::tzOffset for -05:30 is -330(min), but agent is runnning with
tzOffset -270(min)----- resetTZ failed.
The agentTZRegion in:
/rfes/oracle/OEM10g/agent10g/sysman/config/emd.properties is not in agreement with what the agent thinks it should be.Fix your environment.Pick a TZ value that corresponds to time zone settings listed in:
/rfes/oracle/OEM10g/agent10g/sysman/admin/supportedtzs.lst

-  The <AGENT_HOME>/sysman/log/emagent.nohup shows:
---- Sun May 3 17:28:00 2009::The agentTZRegion value in /rfes/oracle/OEM10g/agent10g/sysman/config/emd.properties is not in agreement with what agent thinks it should be.Please verify your environment to make sure that TZ setting has not changed since the last start of the agent.
If you modified the timezone setting in the environment, please stop the agent and exectute 'emctl resetTZ agent' and also execute the script mgmt_target.set_agent_tzrgn(<agent_name>, <new_tz_rgn>) to get the value propagated to repository -----
----- Sun May 3 17:31:24 2009::tzOffset for -04:00 is -240(min), but agent is runnning with tzOffset -270(min)
-----
----- Sun May 3 17:31:24 2009::trying again after waiting for 1 sec to account for daylight transition
-----
----- Sun May 3 17:31:24 2009::tzOffset for -04:00 is -240(min), but agent is runnning with tzOffset -270(min)

-  The <AGENT_HOME>/sysman/log/emdctl.trc shows:

2009-05-03 16:21:56 Thread-1 ERROR main: nmectla_agentctl: Error connecting to https://agentmachine.domain:3872/emd/main/. Returning status code 1
2009-05-03 16:21:56 Thread-1 ERROR main: nmectl.c: nmectl_validateTZRegion, agentTZoffset =-270,and testTZoffset for -05:30:-330 do not match
2009-05-03 16:21:57 Thread-1 ERROR main: nmectl.c: nmectl_validateTZRegion, agentTZoffset =-270,and testTZoffset for -05:30:-330 do not matc

- Value of TZ parameter in the <AGENT_HOME>/sysman/config/emd.properties file is:

agentTZRegion=GMT-05:30
Cause
The Agent machine is located geographically in Asia/Calcutta (GMT+5:30) timezone but the wrong timezone was set at OS level:

$ echo $TZ
IST05:30IST


With the above setting, the 'emctl config agent getTZ' command returns:

emctl config agent getTZ
Oracle Enterprise Manager 10g Release 5 Grid Control 10.2.0.5.0.
Copyright (c) 1996, 2009 Oracle Corporation. All rights reserved.
GMT-05:30

GMT-05:30 is an incorrect value for the Asia/Calcutta timezone.
Solution

On AIX servers the timezone at OS level is set in a different format when compared to the other Operating systems.

1. Set the TZ in the correct format.

For example, if the TZ=Asia/Calcutta, then

$ export TZ=IST-5:30IST

2. Check the TZ value as calculated by the Agent:

emctl config agent getTZ
Oracle Enterprise Manager 10g Release 5 Grid Control 10.2.0.5.0.
Copyright (c) 1996, 2009 Oracle Corporation. All rights reserved.
GMT+05:30

3. Force the Agent to recognize the newly set TZ:
$ export ORACLE_HOME=<path to the agent10g installation>
$ export PATH=$ORACLE_HOME/bin:$PATH
$ cd $ORACLE_HOME/bin
$ emctl resetTZ agent
Oracle Enterprise Manager 10g Release 5 Grid Control 10.2.0.5.0.
Copyright (c) 1996, 2009 Oracle Corporation. All rights reserved.
Updating /home/em/oracle/gc102/agent10g/sysman/config/emd.properties...
Successfully updated /home/em/oracle/gc102/agent10g/sysman/config/emd.properties.
Login as the em repository user and run the script:
exec mgmt_target.set_agent_tzrgn('agentmachine.domain:3872','Asia/Calcutta')
and commit the changes
This can be done for example by logging into sqlplus and doing
SQL> exec mgmt_target.set_agent_tzrgn('agentmachine.domain:3872','Asia/Calcutta')
SQL> commit

DO NOT start the Agent yet!

- Login to the Repository database as the SYSMAN user and execute the SQL statements shown by the above command output:

SQL> exec mgmt_target.set_agent_tzrgn('<agent_name>','<correct TZ region>');
SQL> commit;

For example:
SQL> exec mgmt_target.set_agent_tzrgn('agentmachine.domain:3872','Asia/Calcutta')
SQL> commit

- After the Repository side routine has completed successfully, start the agent, and make sure it's uploading new data:

$ emctl start agent
$ emctl upload
$ emctl status agent