vmware and bridged host

Hello,
I'm using vmware (and will soon try xen) to simulate a multi pc system
for ditributed filesystems tests.

I'm not having difficulties with vmware natted network but for some
guestos i'd like to use bridged network. i.e. my guest os would use the
vmnet0 interface bridged on eth0, so to reach outside network with his
own public ip.

The problem is:
tcpdump on eth0 shows some outgoing activity originating from guestos
public ip, but this traffic is never sent on the wire. I'm tcpdumping
on target machine but no incoming traffis is seen.

I wonder if the tcp/ip stack discards all packet that do not come from
the ip of a local interface.
But if this is the case, how is vmware bridging supposed to work?

I checked if vmware uses linux-bridging tools using the bt config tool,
but it doesn't. They use some other proprietary technology.

Host os are a Fedora Core 5 and a Scientific Linux 4.3 (based on Red
Hat Enterprise 4).
Vmware is vmware-server (two weeks ago version).


Matteo

On Wed, 20 Sep 2006 14:06:04 -0700, falesca wrote:

> Hello,
> I'm using vmware (and will soon try xen) to simulate a multi pc system
> for ditributed filesystems tests.
>
> I'm not having difficulties with vmware natted network but for some
> guestos i'd like to use bridged network. i.e. my guest os would use the
> vmnet0 interface bridged on eth0, so to reach outside network with his
> own public ip.
>
> The problem is:
> tcpdump on eth0 shows some outgoing activity originating from guestos
> public ip, but this traffic is never sent on the wire. I'm tcpdumping
> on target machine but no incoming traffis is seen.
>
> I wonder if the tcp/ip stack discards all packet that do not come from
> the ip of a local interface.
> But if this is the case, how is vmware bridging supposed to work?
>
> I checked if vmware uses linux-bridging tools using the bt config tool,
> but it doesn't. They use some other proprietary technology.
>
> Host os are a Fedora Core 5 and a Scientific Linux 4.3 (based on Red
> Hat Enterprise 4).
> Vmware is vmware-server (two weeks ago version).
>
>
> Matteo

I've run vmplayer, vmware workstation, and vmware server.
I've got them running on 6 different servers, all using
bridged networking. No issues on any of them.

all I've ever had to do was run vmware-config.pl, make
sure that the vmnet-bridge process starts and it's all good.

the guest is configured to use the pcnet32 ethernet device,
sets up for DHCP or a static IP Address and it routes
directly on the hosts network.

My hosts has ( private network ) an address of 192.168.1.1.
The vmware guests have addresses of 192.168.1.2 and 192.168.1.5.
arp shows a different mac address for each IP Address.

I assume your stuff is setup in the same network address space....

jack


--
D.A.M. - Mothers Against Dyslexia

see http://www.jacksnodgrass.com for my contact info.

jack - Grapevine/Richardson

falesca wrote:
> Hello,
> I'm using vmware (and will soon try xen) to simulate a multi pc system
> for ditributed filesystems tests.
>
> I'm not having difficulties with vmware natted network but for some
> guestos i'd like to use bridged network. i.e. my guest os would use the
> vmnet0 interface bridged on eth0, so to reach outside network with his
> own public ip.
>
> The problem is:
> tcpdump on eth0 shows some outgoing activity originating from guestos
> public ip, but this traffic is never sent on the wire. I'm tcpdumping
> on target machine but no incoming traffis is seen.
>
> I wonder if the tcp/ip stack discards all packet that do not come from
> the ip of a local interface.
> But if this is the case, how is vmware bridging supposed to work?
>
> I checked if vmware uses linux-bridging tools using the bt config tool,
> but it doesn't. They use some other proprietary technology.
>
> Host os are a Fedora Core 5 and a Scientific Linux 4.3 (based on Red
> Hat Enterprise 4).
> Vmware is vmware-server (two weeks ago version).
>
>
> Matteo
>

Do you have any firewall/packet filtering in place on the host ?

B.

> I've run vmplayer, vmware workstation, and vmware server.
> I've got them running on 6 different servers, all using
> bridged networking. No issues on any of them.
>
> all I've ever had to do was run vmware-config.pl, make
> sure that the vmnet-bridge process starts and it's all good.
>
> the guest is configured to use the pcnet32 ethernet device,
> sets up for DHCP or a static IP Address and it routes
> directly on the hosts network.

pcnet32 with fixed address

>
> My hosts has ( private network ) an address of 192.168.1.1.
> The vmware guests have addresses of 192.168.1.2 and 192.168.1.5.
> arp shows a different mac address for each IP Address.
>
> I assume your stuff is setup in the same network address space....
>

yes they are in the same network address space (but i don't see why it
should be a problem to have host and guest on different networks as
they are bridged, maybe apart from talking to each other)

I ping'ed from guest machine an external machine, but i don't receive
any icmp message . I have a switched network so it's not a routing
problem.

But it occours to me now that i could check the switch forwarding
tables of the switch to see if my guest os mac address is registered.

There is no firewall or selinux active

I just noticed that there is no vmnet0 device appearing with ifconfig
on the host, but i thought it was normal because the pcnet32 interface
is bridged on the eth0 of the host.

On Fri, 22 Sep 2006 13:02:22 -0700, falesca wrote:

>> I've run vmplayer, vmware workstation, and vmware server.
>> I've got them running on 6 different servers, all using
>> bridged networking. No issues on any of them.
>>
>> all I've ever had to do was run vmware-config.pl, make
>> sure that the vmnet-bridge process starts and it's all good.
>>
>> the guest is configured to use the pcnet32 ethernet device,
>> sets up for DHCP or a static IP Address and it routes
>> directly on the hosts network.
>
> pcnet32 with fixed address
>
>>
>> My hosts has ( private network ) an address of 192.168.1.1.
>> The vmware guests have addresses of 192.168.1.2 and 192.168.1.5.
>> arp shows a different mac address for each IP Address.
>>
>> I assume your stuff is setup in the same network address space....
>>
>
> yes they are in the same network address space (but i don't see why it
> should be a problem to have host and guest on different networks as
> they are bridged, maybe apart from talking to each other)
>
> I ping'ed from guest machine an external machine, but i don't receive
> any icmp message . I have a switched network so it's not a routing
> problem.
>
> But it occours to me now that i could check the switch forwarding
> tables of the switch to see if my guest os mac address is registered.
>
> There is no firewall or selinux active
>
> I just noticed that there is no vmnet0 device appearing with ifconfig
> on the host, but i thought it was normal because the pcnet32 interface
> is bridged on the eth0 of the host.

no net devices will register...

ps -aef | grep vmnet-bridge
should show you something like:
root 2492 1 0 Sep20 ? 00:00:00 /usr/bin/vmnet-bridge -d /var/run/vmnet-bridge-0.pid /dev/vmnet0 eth0
if the device is not present when the guest boots, it will
detect it and give you an error. When you guest boots, you
should see an icon for the ethernet device and be able to
disable it or enable it.

you should be able to run tcpdump on both the guest
and the host for the 'briged' ethernet device and see
the systems sending data.

arp -n on both the host and guest should show each others
info...

arp -n | grep eth0
192.168.1.25 ether 00:12:17:FD:CC:70 C eth0
192.168.1.5 ether 00:0C:29:3E:E2:35 C eth0
192.168.1.2 ether 00:0C:6E:4B:1F:70 C eth0
192.168.1.15 ether 00:50:8D:86:07:4D C eth0
192.168.1.21 ether 00:0C:29:C4:AC:90 C eth0

the .5 and .21 boxes are vmware guests....

on a vmware guests....
arp -n
Address HWtype HWaddress Flags Mask Iface
192.168.1.15 ether 00:50:8D:86:07:4D C eth0
192.168.1.1 ether 00:133:C14:51 C eth0

for grins... on the guest...
ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:0C:29:C4:AC:90
inet addr:192.168.1.21 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fec4:ac90/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:440184 errors:0 dropped:0 overruns:0 frame:0
TX packets:414161 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:131452843 (125.3 MiB) TX bytes:76571523 (73.0 MiB)
Interrupt:177 Base address:0x1080

[root@websrvr ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 eth0

In my case... the host ( 192.168.1.1 ) is also the firewall, but my
guest does not have issues talking to other boxes in my network.

Not sure what else it can be other than an iptables issue.

jack

--
D.A.M. - Mothers Against Dyslexia

see http://www.jacksnodgrass.com for my contact info.

jack - Grapevine/Richardson

Sorry for not answering before, but I could not work on this project
for a while.

Jack Snodgrass ha scritto:

> On Fri, 22 Sep 2006 13:02:22 -0700, falesca wrote:
>
> >> I've run vmplayer, vmware workstation, and vmware server.
> >> I've got them running on 6 different servers, all using
> >> bridged networking. No issues on any of them.
> >>
> >> all I've ever had to do was run vmware-config.pl, make
> >> sure that the vmnet-bridge process starts and it's all good.
> >>
> >> the guest is configured to use the pcnet32 ethernet device,
> >> sets up for DHCP or a static IP Address and it routes
> >> directly on the hosts network.
> >
> > pcnet32 with fixed address
> >
> >>
> >> My hosts has ( private network ) an address of 192.168.1.1.
> >> The vmware guests have addresses of 192.168.1.2 and 192.168.1.5.
> >> arp shows a different mac address for each IP Address.
> >>
> >> I assume your stuff is setup in the same network address space....
> >>
> >
> > yes they are in the same network address space (but i don't see why it
> > should be a problem to have host and guest on different networks as
> > they are bridged, maybe apart from talking to each other)

Here there may be the issue... se below.

> >
> > I ping'ed from guest machine an external machine, but i don't receive
> > any icmp message . I have a switched network so it's not a routing
> > problem.
> >
> > But it occours to me now that i could check the switch forwarding
> > tables of the switch to see if my guest os mac address is registered.
> >
> > There is no firewall or selinux active
> >
> > I just noticed that there is no vmnet0 device appearing with ifconfig
> > on the host, but i thought it was normal because the pcnet32 interface
> > is bridged on the eth0 of the host.
>
> no net devices will register...
>
> ps -aef | grep vmnet-bridge
> should show you something like:
> root 2492 1 0 Sep20 ? 00:00:00 /usr/bin/vmnet-bridge -d /var/run/vmnet-bridge-0.pid /dev/vmnet0 eth0
> if the device is not present when the guest boots, it will
> detect it and give you an error. When you guest boots, you
> should see an icon for the ethernet device and be able to
> disable it or enable it.
>

This is present
root 6311 0.0 0.0 1300 220 pts/1 S 12:57 0:00
/usr/bin/vmnet-bridge -d /var/run/vmnet-bridge-0.pid /dev/vmnet0 eth0


> you should be able to run tcpdump on both the guest
> and the host for the 'briged' ethernet device and see
> the systems sending data.
>

I see in fact the icmp traffic on eth0 of the host and the guest

> arp -n on both the host and guest should show each others
> info...
>
> arp -n | grep eth0
> 192.168.1.25 ether 00:12:17:FD:CC:70 C eth0
> 192.168.1.5 ether 00:0C:29:3E:E2:35 C eth0
> 192.168.1.2 ether 00:0C:6E:4B:1F:70 C eth0
> 192.168.1.15 ether 00:50:8D:86:07:4D C eth0
> 192.168.1.21 ether 00:0C:29:C4:AC:90 C eth0
>
> the .5 and .21 boxes are vmware guests....
>
> on a vmware guests....
> arp -n
> Address HWtype HWaddress Flags Mask Iface
> 192.168.1.15 ether 00:50:8D:86:07:4D C eth0
> 192.168.1.1 ether 00:133:C14:51 C eth0
>
> for grins... on the guest...
> ifconfig eth0
> eth0 Link encap:Ethernet HWaddr 00:0C:29:C4:AC:90
> inet addr:192.168.1.21 Bcast:192.168.1.255 Mask:255.255.255.0
> inet6 addr: fe80::20c:29ff:fec4:ac90/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:440184 errors:0 dropped:0 overruns:0 frame:0
> TX packets:414161 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:131452843 (125.3 MiB) TX bytes:76571523 (73.0 MiB)
> Interrupt:177 Base address:0x1080
>

about the same thing

> [root@websrvr ~]# route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use Iface
> 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
> 0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 eth0
>
> In my case... the host ( 192.168.1.1 ) is also the firewall, but my
> guest does not have issues talking to other boxes in my network.
>

The problem could be here.
I used as gateway for the def route the gateway of the subnet... that
is NOT the host
The host is 192.168.21.23 and the gateway is an external router with ip
192.168.21.254

Being the interfaces of guest-host bridged together (layer 2 link), I
thought the host doesn't have to be a router for the guest os.
am I wrong?

> Not sure what else it can be other than an iptables issue.
iptables is off for sure

> > [root@websrvr ~]# route -n
> > Kernel IP routing table
> > Destination Gateway Genmask Flags Metric Ref Use Iface
> > 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
> > 0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 eth0
> >
> > In my case... the host ( 192.168.1.1 ) is also the firewall, but my
> > guest does not have issues talking to other boxes in my network.
> >
>
> The problem could be here.
> I used as gateway for the def route the gateway of the subnet... that
> is NOT the host
> The host is 192.168.21.23 and the gateway is an external router with ip
> 192.168.21.254
>

The problem was there.
net.ipv4.forward must be set to 1
and the gateway must be the host public interface (the bridged one)
Actually i didn't find any page of the vmware server doc that listed
this two requirements. I didn't imagine that the bridged network was a
mix of routing and bridging. I understand that packets are routed
toward the outside and bridged to the internal interface (in part
filtered maybe) from the outside to the inside when they are received
by the host.

Thank you,
Matteo