Linux – Primary network interface in Linux

interfacelinuxnetworkingrouting

I have a server box with two interfaces.
One of the interfaces is connected to the internal network, another – to external.

I have assigned IP 192.168.1.200 to internal interface (eth0). I have some local daemons listening on this IP/interface (cups, nginx, pdns).

External IP is 192.168.0.91 (eth1). Here I have only NAT masquerading.
Strange things happens whenever I'm trying to access server box from itself.

If I'm trying to open webpage, located on 192.168.1.200, server uses 192.168.0.91 address and I'm getting permission denied (webserver protected to serve content on internal network only).

If I'm bring down eth1, server work OK and uses internal IP. But, as soon, as I will bring up eth1 it will immediately pick up it's IP as primary one and I will get permission denied again.

How can I explicitly set main IP/interface to use?

I'm running 64-bit version of Gentoo Linux.
Both nic's drivers are compiled as modules.
I'm using systemd as my init system.

EDIT:

Thanks for responses, but most interesting part is coming here:

atomic ~ # cat /etc/resolv.conf
domain local
search local
nameserver 192.168.1.200

atomic ~ # cat /etc/hosts
...
192.168.1.200   atomic ns.atomic.local atomic.local
...

Address resolving just OK:

atomic ~ # dig atomic.local 192.168.1.200

; <<>> DiG 9.9.4 <<>> atomic.local
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 38797
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 2800
;; QUESTION SECTION:
;atomic.local.          IN  A

;; ANSWER SECTION:
atomic.local.       604800  IN  A   192.168.1.200

;; Query time: 42 msec
;; SERVER: 192.168.1.200#53(192.168.1.200)
;; WHEN: Tue May 27 13:37:04 EEST 2014
;; MSG SIZE  rcvd: 55

And Nginx logs of accessing via wget:

atomic ~ # wget atomic.fhn
--2014-05-27 13:45:58--  http://atomic.local/
Resolving atomic.local... 192.168.1.200
Connecting to atomic.local|192.168.1.200|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘index.html’

[ <=>                                              ] 0  --.-K/s   in 0s      

2014-05-27 13:45:58 (0.00 B/s) - ‘index.html’ saved [0]

atomic ~ # tail -n 1 /var/log/nginx/access_log
192.168.0.91 - - [27/May/2014:13:45:58 +0300] "GET / HTTP/1.1" 200 5 "-" "Wget/1.14 (linux-gnu)"

I got 200/OK HTTP status, because I have disabled IP/network filtering while problem is not solved.

Routing table:

192.168.0.0/30 dev wan  proto kernel  scope link  src 192.168.0.2 
192.168.1.0/24 dev lan  proto kernel  scope link  src 192.168.1.200 

EDIT2:

Routing table with metric used:

192.168.0.0/30 dev wan  scope link  metric 20 
192.168.1.0/24 dev lan  scope link  metric 10 
broadcast 127.0.0.0 dev lo  table local  proto kernel  scope link  src 127.0.0.1 
local 127.0.0.0/8 dev lo  table local  proto kernel  scope host  src 127.0.0.1 
local 127.0.0.1 dev lo  table local  proto kernel  scope host  src 127.0.0.1 
broadcast 127.255.255.255 dev lo  table local  proto kernel  scope link  src 127.0.0.1 
broadcast 192.168.0.0 dev wan  table local  proto kernel  scope link  src 192.168.0.2 
local 192.168.0.2 dev wan  table local  proto kernel  scope host  src 192.168.0.2 
broadcast 192.168.0.3 dev wan  table local  proto kernel  scope link  src 192.168.0.2 
broadcast 192.168.1.0 dev lan  table local  proto kernel  scope link  src 192.168.1.200 
local 192.168.1.200 dev lan  table local  proto kernel  scope host  src 192.168.1.200 
broadcast 192.168.1.255 dev lan  table local  proto kernel  scope link  src 192.168.1.200 

Best Answer

What you need is to make sure the LAN route takes precedence over the WAN one. The trouble is here:

192.168.0.0/30 dev wan  proto kernel  scope link  src 192.168.0.2 
192.168.1.0/24 dev lan  proto kernel  scope link  src 192.168.1.200

192.168.0.0/30 covers 192.168.1.0/24 as well. Hence you have to tell the TCP/IP stack to prefer the latter when applicable otherwise it might choose randomly (I would expect most implementations to always use the first one) - use the metric argument, e.g.:

ip route add 192.168.1.0/24 dev lan metric 10
ip route add 192.168.0.0/30 dev wan metric 20

(you'll obviously need to remove the extant routes first).

Related Question