refuse in notify slave

[Nelson Serafica]

I have multiple ip address on my primary ns server. (eth0 , eth0:1 , eth0:2). Let's say eth0 is 1.2.3.4, eth0:1 is 
2.3.4.5 and th0:2 is 3.4.5.6. I have a slave ns server but everytime I do rndc reload and check secondary ns on syslog, 
I see

refused notify from non-master: 1.2.3.4#48499

where 1.2.3.4 is the ip of eth0. Is it possible the ip address that will send to slave will be 4.5.6.7 (eth0:2) and not 
1.2.3.4 (eth0)?