You are viewing lathiat

Lathiat's Journal - Watch out for hostname changes when using replication! [entries|archive|friends|userinfo]
Trent 'Lathiat' Lloyd (トレント)

[ website | lathiat dot net ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Watch out for hostname changes when using replication! [Aug. 25th, 2009|05:40 pm]
Previous Entry Share Next Entry
Cross-post from my blog at sun

For one reason or another, many times we find ourselves changing the hostname of a machine. It's been repurposed or moved - or perhaps the original installer didn't know what name it should have. To achieve this on most modern Linux distributions there are 2 key files you need to update.


  1. /etc/hostname needs to be updated with correct hostname to be set on boot, and

  2. /etc/hosts needs to be updated for DNS lookups of the local hostname. This is more important than you might think and will break many applications if not updated.



Some people also take the third step of updating the hostname on the fly with the 'hostname' tool, which if you do that means the gotcha I'm about to describe take you completely unaware in some weeks or months.

If you are using MySQL replication, there are two key options which depend on the hostname. These are the 'log-bin' and 'relay-log' for the binary log and binary log replication log paths respectively. The problem is not only do the logs themselves depend on the hostname, so does the index which tells you where to find them. So if you restart the server, it will look for a new index file and won't find it - causing errors such as:

090825 17:17:15 [ERROR] Failed to open the relay log './mellie-relay-bin.000002' (relay_log_pos 339)
090825 17:17:15 [ERROR] Could not find target log during relay log initialization


There are several possible solutions to this, one involves combining the old and new files (which you can find documented here - but that's a more pro-active approach. The second is to completely restart the replication process - which in my opinion, is cleaner. So I will detail that approach.

First we need to stop the slave process, just to make sure.

mysql> STOP SLAVE;


Then we can get the slave status, to see what position the master is currently. You can see what that looks like here. The important values to note are the following two:

mysql> SHOW SLAVE STATUS\G
*************************** 1. row ***************************
Master_Host: localhost
Master_User: root
Master_Port: 3306
Relay_Master_Log_File: gbichot-bin.005
Exec_Master_Log_Pos: 79


These values tell us what the current position in the master's binary logs the slave has executed up to as well as the basic master details. The reason this is important is we are going to tell the slave to completely forget about it's current replication information and fetch the data fresh.

While normally you could just change the master log file and position, since it can't open the relay log at all - the slave replication does not start and we must completely reset and specify all of the details again. The above information contains everything you need except the password for the replication user. You can find that by reading either the 'master.info' file or by prior knowledge.

ERROR 1201 (HY000): Could not initialize master info structure; more error messages can be found in the MySQL error log


To reset that, we run RESET SLAVE like so:

mysql> RESET SLAVE;


Then we run we need to construct a CHANGE MASTER statement with the above information.

In our case the statement is
mysql> RESET SLAVE;
mysql> CHANGE MASTER TO MASTER_HOST="localhost", MASTER_USER="root", MASTER_PASSWORD="test", MASTER_LOG_FILE="gbichot-bin.005", MASTER_LOG_POS=79;

... but be sure to use your own values from the SLAVE STATUS and make sure the log and position are from Relay_Master_Log_File and Exec_Master_Log_Pos - there are other values that look similar so don't confuse them. Once this is done we can start the slave, and check the status to make sure it is replicating correctly.

mysql> START SLAVE;
*wait a few moments*
mysql> SHOW SLAVE STATUS\G


When the slave status is displayed, make sure that both the IO and SQL threads are running and there are no errors.


mysql> SHOW SLAVE STATUS\G
*************************** 1. row ***************************
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Seconds_Behind_Master: 8


Also keep an eye on the Seconds_Behind_Master value and make sure that it is reducing to 0.


That's this weeks situation. If you do change your hostname, it pays to reboot your machine to make sure everything has worked and that issues will not pop up down the track when your machine is unexpectedly rebooted by a crash or some other circumstance. You don't need unexpected changes causing problems!

This applies to any situation where you might change a configuration file, but making it's effects current are done separately. This is very common in MySQL, where you might want to change a dynamic variable and also edit the my.cnf file. If you make a syntax error, you won't know until your server reboots. So it helps to be very careful and preferably do the actual MySQL or server reboot.
LinkReply

Comments:
From: (Anonymous)
2011-01-17 12:53 pm (UTC)

fvtbbygy nokh

(Link)

yghr aqzwv xnxx (http://www.miasporn.com) fnedot c hu v orr
From: (Anonymous)
2011-01-19 04:16 pm (UTC)

Hi-ya every one, neat website

(Link)

Good Day everyone, decent message board I have found It vastly useful and it's helped me loads
I hope to contribute & guide other people like this message board has helped me

_________________
[url=http://iphoneusers.com]unlock iphone 4.2.1[/url]
From: (Anonymous)
2011-01-26 03:29 pm (UTC)

dog12 dog clothes wholesale gb

(Link)

House-broken dogs inherited a complex dog clothes wholesale group hierarchy and behaviors from their wolf ancestors. Dogs are peck animals with a complex lay of behaviors cognate dog clothes wholesale to determining each dog's position in the collective hierarchy, and they exhibit several postures and other means of nonverbal communication that reveal their states of mind. dog clothes wholesale These sophisticated forms of societal cognition and communication may account for their trainability, playfulness, and power to adjust into kind-hearted households dog clothes wholesale and social situations, and these attributes have earned dogs a in perfect accord relationship with humans despite being potentially rickety apex predators. http://www.lovelonglong.com Would like to unsubscribe from this business, please send your URL to the E-mail: unsubscribe@1yingxiao.com to unsubscribe
From: (Anonymous)
2011-04-08 04:12 am (UTC)

NATO takes over command of military operations in Libya

(Link)

[b]NATO is taking over command of military operations in Libya from coalition forces, world media reported Sunday.[/b]

The UN Security Council imposed the no-fly zone over Libya on March 17, along with ordering "all necessary measures" to protect civilians from Muammar Gaddafi's attacks on rebel-held towns.

The 28 NATO ambassadors met on Sunday to decide on further military plans in Libya.

The United States transfers command for a no-fly zone over Libya to NATO, while coalition forces will continue to protect civilian population from attacks by Gaddafi forces.

The military operation in Libya, codenamed Odyssey Dawn, has been conducted so far jointly by 13 states, including the United States, Britain and France.

NATO members decided on Thursday to assume responsibility for the enforcement of a no-fly zone in Libya, but could not agree on taking full command of all military operations in the country.

Meanwhile, leaders of the 27 European Union states on Thursday issued a statement saying the EU stood ready to assist in building a new Libya "in cooperation with the United Nations, the Arab League, the African Union and others."

MOSCOW, March 27 (RIA Novosti)

http://en.rian.ru/world/20110327/163235937.html
From: witefoam
2011-04-10 08:53 am (UTC)

(Link)

Great post! I want to see a follow up to this topic

From: witefoam
2011-04-12 08:26 am (UTC)

(Link)

Great writing! I want you to follow up to this topic!?!

From: mogurama
2011-04-15 05:03 am (UTC)

(Link)

What a great resource!