Hi Friends,

Last entire week i was busy playing with Oracle Enterprise Manager Grid Control alongside Siebel, OBIEE & SOA Management Packs.

Believe me, its not a piece of cake!
Though PSU 3 on top of 10.2.0.5 provides great stability, still few nagging issues around Metric collections for different target types & other areas keep affecting us.

One of the most annoying issue that took me long time to figure out was my Grid Control Agent, simply failing to startup :-(

It was working perfectly fine earlier, but didn't know when it stopped working :-(
After much due diligence in the logs, the emagent.nohup had following dumps:
Issue:
----- Wed Jun 23 11:32:51 2010::Agent Launched with PID 3852 at time Wed Jun 23 11:32:51 2010 -----
(pid=3852): starting emagent version 10.2.0.5.0
(pid=3852): emagent started successfully
OMS decided to shutdown the agent because of the following reason sent from OMS: AGENT_TZ_MISMATCH(pid=3852): emagent now exiting abnormally - initialization failure. Consult '.trc' and '.log' files.
----- Wed Jun 23 11:34:39 2010::Pid 3852 not found. reaped=3852, processStatus=14080 -----
----- Wed Jun 23 11:34:40 2010::Checking status of EMAgent : 3852 -----
----- Wed Jun 23 11:34:40 2010::EMAgent exited at Wed Jun 23 11:34:40 2010 with return value 55. -----
----- Wed Jun 23 11:34:40 2010::EMAgent has exited due to initialization failure. -----
----- Wed Jun 23 11:34:40 2010::Stopping other components. -----
----- Wed Jun 23 11:34:40 2010::Commiting Process death. -----
----- Wed Jun 23 11:34:40 2010::Exiting watchdog loop

I was scratching my head, what does AGENT_TZ_MISMATCH means?? Googled it around no luck! I was shocked & surprised that there was no results in Google!

Reason:
Felt like i have hit some kind of jackpot ;-)
After much research, i came to know that such kind of issue might happen when your Agent's host has its Time Zone changed.

Now, first of all, i didnt accept it! What does Agent's Timezone has to do with its startup? It should automatically accept the new time...

But then, it was clear that Agent did startup correctly, but when it use to communicate with my OMS Server where the Agent's Timezone is recorded, the changed TimeZone hits a mismatch here!!!

Solution:
Well, in order to resolve this, the solution is simple!
From a command prompt or terminal go to %Agent_Home%/bin directory.
Execute below command:

prompt:/>emctl resetTZ agent - Windows
bash-$./emctl resetTZ agent - Unix/Linux

This will update your Agent's emd.properties with the changed TimeZone. But wait this does not complete the changes. You will need to execute a procedure on your Enterprise Manager Repository Database to reflect the changes in the timezone of the Agent.

So connect to your OMS Repository Database (generally 'emrep') using the repository owner (generally 'sysman')

SQL>exec mgmt_target.set_agent_tzrgn(<agent_name>, <new_tz_rgn>);

Well, the exact statement for this procedure is automatically generated at the end of 'emctl resetTZ agent' command :-)

Well, this completes your Agent's TimeZone change. Now to try to start the agent using below command:

prompt:/>emctl start agent - Windows
bash-$./emctl start agent - Unix/Linux

I am sure it will startup correctly! ;-)

Keywords:
OEM, OMS, Enterprise Manager, Grid Control, Grid Control Agent, OMS Agent, OBIEE Management Pack, Siebel Management Pack, SOA Management Pack, AGENT_TZ_MISMATCH, Timezone, 10.2.0.5, PSU 3, etc