Re: Asymmetric TCP/ATM throughputs between client and server

Edwin Law (e-law@cs.yorku.ca)
Mon, 30 Mar 1998 11:46:40 -0500 (EST)

This message is in MIME format. The first part should be readable text,
while the remaining parts are likely unreadable without MIME-aware tools.
Send mail to mime@docserver.cac.washington.edu for more info.

---559023410-851401618-891276400=:28712
Content-Type: TEXT/PLAIN; charset=US-ASCII

I am not sure if it is appropriate here, but I have attached here the
small program I used for me tests, in case it is due to a bug in the
program (which is quite probable). The switch driver I am using is 4.1.0.
The Sun's are running Solaris 2.5.1. The Qos parameters are stated in the
program.

Thanks,
Edwin.

On Fri, 27 Mar 1998, Kevin M. Joyce wrote:

> We're running similar tests here and didn't see the same results. We
> get consistent throughput readings from both sides.
>
> Edwin Law wrote:
> >
> > Hi,
> > I wonder if anyone has encountered a similar problem before. I have
> > obtained different throughput readings when transferring data between a
> > client and a server from both directions. I am talking about a different
> > of about 50 times in magnitude. This happens when I use the TCP/IP over
> > ATM and the default ATM MTU of 9188 on two Sun's running Solaris. When I
> > set the ATM MTU to 65535, the problem goes away.
> >

---559023410-851401618-891276400=:28712
Content-Type: TEXT/PLAIN; charset=US-ASCII; name="deadlock.c"
Content-Transfer-Encoding: BASE64
Content-ID: <Pine.GSO.3.96.980330114640.28712D@tiger>
Content-Description: deadlock.c

I2luY2x1ZGUgPHN0ZGlvLmg+DQojaW5jbHVkZSA8ZmNudGwuaD4NCiNpbmNs
dWRlIDxzeXMvdHlwZXMuaD4NCiNpbmNsdWRlIDxzeXMvc29ja2V0Lmg+DQoj
aW5jbHVkZSA8bmV0ZGIuaD4NCiNpbmNsdWRlIDxuZXRpbmV0L2luLmg+DQoj
aW5jbHVkZSA8bmV0aW5ldC90Y3AuaD4NCiNpbmNsdWRlIDxzeXMvdGltZS5o
Pg0KDQojZGVmaW5lIE1BWF9CVUZfU0laRSA2NTUzNg0KDQpzdHJ1Y3QgdGlt
ZXZhbCBzdGFydCwgZmluaXNoOw0KDQp2b2lkIHN0YXJ0X3RpbWVyKCkgew0K
ICAgIGdldHRpbWVvZmRheSgmc3RhcnQsIChzdHJ1Y3QgdGltZXpvbmUqKTAp
Ow0KfQ0KDQpkb3VibGUgc3RvcF90aW1lcigpIHsNCiAgICBsb25nIHRpbWU7
DQogDQogICAgZ2V0dGltZW9mZGF5KCZmaW5pc2gsIChzdHJ1Y3QgdGltZXpv
bmUqKTApOw0KICAgIHRpbWUgPSAoZmluaXNoLnR2X3NlYyAtIHN0YXJ0LnR2
X3NlYykgKiAxMDAwMDAwICsgDQogICAgICAgICAgICBmaW5pc2gudHZfdXNl
YyAtIHN0YXJ0LnR2X3VzZWM7DQogICAgcmV0dXJuIChkb3VibGUpIHRpbWUg
LyAxMDAwMDAwOw0KfQ0KDQp2b2lkIHNlcnZlcihpbnQgcG9ydCkgew0KICAg
IGludCBtYWluZmQsIGZkLCBsZW4sIGJ1Zl9zaXplLCBzaXplLCBuLCB0b3Rh
bDsNCiAgICBjaGFyIGJ1ZltNQVhfQlVGX1NJWkVdOw0KICAgIGRvdWJsZSB0
aW1lOw0KICAgIHN0cnVjdCBzb2NrYWRkcl9pbiBzZXJ2ZXIsIGNsaWVudDsN
Cg0KLyogU29ja2V0IHNlcnZlciBzZXQgdXAgLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0gKi8NCiAgICBp
ZigobWFpbmZkID0gc29ja2V0KEFGX0lORVQsIFNPQ0tfU1RSRUFNLCAwKSkg
PT0gLTEpIHsNCiAgICAgICAgcGVycm9yKCJzb2NrZXQiKTsNCiAgICAgICAg
ZXhpdCgxKTsNCiAgICB9DQogICAgc2VydmVyLnNpbl9mYW1pbHkgPSBBRl9J
TkVUOw0KICAgIHNlcnZlci5zaW5fYWRkci5zX2FkZHIgPSBJTkFERFJfQU5Z
Ow0KICAgIHNlcnZlci5zaW5fcG9ydCA9IGh0b25zKHBvcnQpOw0KICAgIGlm
KGJpbmQobWFpbmZkLCAoc3RydWN0IHNvY2thZGRyICopJnNlcnZlciwgc2l6
ZW9mKHNlcnZlcikpID09IC0xKSB7DQogICAgICAgIHBlcnJvcigiYmluZCIp
Ow0KICAgICAgICBleGl0KDEpOw0KICAgIH0NCi8qIC4uLg0KICAgIHNldHNv
Y2tvcHQobWFpbmZkLCBnZXRwcm90b2J5bmFtZSgidGNwIiktPnBfcHJvdG8s
IFRDUF9OT0RFTEFZLA0KICAgICAgICAgICAgICAoY2hhciAqKSZvbmUsIHNp
emVvZihvbmUpKTsNCi4uLiAqLw0KICAgIGxpc3RlbihtYWluZmQsIDEpOw0K
ICAgIGZwcmludGYoc3RkZXJyLCAiXG5TZXJ2ZXIgaXMgbGlzdGVuaW5nIGZy
b20gcG9ydCAlZCAuLi5cbiIsIHBvcnQpOw0KLyogLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0qLw0KICAgIHdoaWxlKDEpIHsNCiAgICAgICAgbGVu
ID0gc2l6ZW9mKGNsaWVudCk7DQogICAgICAgIGlmKChmZCA9IGFjY2VwdCht
YWluZmQsIChzdHJ1Y3Qgc29ja2FkZHIgKikmY2xpZW50LCAmbGVuKSkgPT0g
LTEpIHsNCiAgICAgICAgICAgIHBlcnJvcigiYWNjZXB0Iik7DQogICAgICAg
ICAgICBleGl0KDEpOw0KICAgICAgICB9DQogICAgICAgIHJlYWQoZmQsICZi
dWZfc2l6ZSwgc2l6ZW9mKGludCkpOyAvKiByZWFkIGJ1ZmZlciBzaXplICov
DQogICAgICAgIHJlYWQoZmQsICZzaXplLCBzaXplb2YoaW50KSk7IC8qIHJl
YWQgZGF0YSBzaXplICovDQogICAgICAgIGZwcmludGYoc3RkZXJyLCAiRGF0
YSBidWZmZXIgc2l6ZSBvZiAlZCBieXRlcyAuLi4gIiwgYnVmX3NpemUpOw0K
ICAgICAgICB0b3RhbCA9IDA7DQogICAgICAgIHN0YXJ0X3RpbWVyKCk7DQog
ICAgICAgIGlmKHNpemUgPD0gMCkgeyAvKiBmcm9tIHNlcnZlciB0byBjbGll
bnQgKi8NCiAgICAgICAgICAgIHNpemUgPSAtc2l6ZTsgDQogICAgICAgICAg
ICB3aGlsZSh0b3RhbCA8IHNpemUpIHsNCiAgICAgICAgICAgICAgICBuID0g
KHRvdGFsICsgYnVmX3NpemUgPiBzaXplKSA/IHNpemUgLSB0b3RhbCA6IGJ1
Zl9zaXplOw0KICAgICAgICAgICAgICAgIGlmKHdyaXRlKGZkLCBidWYsIG4p
ICE9IG4pIHsNCiAgICAgICAgICAgICAgICAgICAgcGVycm9yKCJ3cml0ZSIp
Ow0KICAgICAgICAgICAgICAgICAgICBleGl0KDEpOyAgIA0KICAgICAgICAg
ICAgICAgIH0gICANCiAgICAgICAgICAgICAgICB0b3RhbCArPSBuOw0KICAg
ICAgICAgICAgIH0NCiAgICAgICAgICAgICB0aW1lID0gc3RvcF90aW1lcigp
Ow0KICAgICAgICAgICAgIGZwcmludGYoc3RkZXJyLCAic2VudCIpOw0KICAg
ICAgICB9DQogICAgICAgIGVsc2UgeyAvKiBmcm9tIGNsaWVudCB0byBzZXJ2
ZXIgKi8NCiAgICAgICAgICAgIHdoaWxlKChuID0gcmVhZChmZCwgYnVmLCBi
dWZfc2l6ZSkpID4gMCkNCiAgICAgICAgICAgICAgICB0b3RhbCArPSBuOw0K
ICAgICAgICAgICAgdGltZSA9IHN0b3BfdGltZXIoKTsNCiAgICAgICAgICAg
IGZwcmludGYoc3RkZXJyLCAicmVjZWl2ZWQiKTsNCiAgICAgICAgfQ0KICAg
ICAgICBmcHJpbnRmKHN0ZGVyciwgIiAlZCBieXRlcyBpbiAlLjZmIHNlY29u
ZHNcbiIsIHRvdGFsLCB0aW1lKTsNCiAgICAgICAgY2xvc2UoZmQpOw0KICAg
IH0gICAgDQp9DQoNCnZvaWQgY2xpZW50KGNoYXIgKnNlcnZlcm5hbWUsIGlu
dCBwb3J0LCBpbnQgYnVmX3NpemUsIGludCBzaXplKSB7DQogICAgaW50IGZk
LCBuLCB0b3RhbDsNCiAgICBjaGFyIGJ1ZltNQVhfQlVGX1NJWkVdOw0KICAg
IGRvdWJsZSB0aW1lOw0KICAgIHN0cnVjdCBzb2NrYWRkcl9pbiBzZXJ2ZXI7
DQogICAgc3RydWN0IGhvc3RlbnQgKmhvc3Q7DQoNCi8qIFNvY2tldCBjbGll
bnQgc2V0IHVwIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tICovDQogICAgaWYoKGZkID0gc29ja2V0KEFG
X0lORVQsIFNPQ0tfU1RSRUFNLCBJUFBST1RPX1RDUCkpID09IC0xKSB7DQog
ICAgICAgIHBlcnJvcigic29ja2V0Iik7DQogICAgICAgIGV4aXQoMSk7ICAN
CiAgICB9DQogICAgaWYoKGhvc3QgPSBnZXRob3N0YnluYW1lKHNlcnZlcm5h
bWUpKSA9PSBOVUxMKSB7DQogICAgICAgIHBlcnJvcigiZ2V0aG9zdGJ5bmFt
ZSIpOw0KICAgICAgICBleGl0KDEpOyAgDQogICAgfQ0KICAgIHNlcnZlci5z
aW5fZmFtaWx5ID0gQUZfSU5FVDsNCiAgICBzZXJ2ZXIuc2luX2FkZHIuc19h
ZGRyID0gSU5BRERSX0FOWTsNCiAgICBzZXJ2ZXIuc2luX3BvcnQgPSBodG9u
cyhwb3J0KTsNCiAgICBtZW1jcHkoKGNoYXIgKikmc2VydmVyLnNpbl9hZGRy
LCBob3N0LT5oX2FkZHIsIGhvc3QtPmhfbGVuZ3RoKTsNCiAgICBpZihjb25u
ZWN0KGZkLCAoc3RydWN0IHNvY2thZGRyICopJnNlcnZlciwgc2l6ZW9mKHNl
cnZlcikpKSB7DQogICAgICAgIHBlcnJvcigiY29ubmVjdCIpOw0KICAgICAg
ICBleGl0KDEpOw0KICAgIH0NCiAgICBmcHJpbnRmKHN0ZGVyciwgIlxuQ29u
bmVjdGVkIHRvICVzKCVkKVxuIiwgc2VydmVybmFtZSwgcG9ydCk7DQovKiAt
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLSovDQogICAgaWYod3JpdGUo
ZmQsICZidWZfc2l6ZSwgc2l6ZW9mKGludCkpICE9IHNpemVvZihpbnQpKSB7
DQogICAgICAgIHBlcnJvcigid3JpdGUiKTsNCiAgICAgICAgZXhpdCgxKTsg
IA0KICAgIH0NCiAgICBpZih3cml0ZShmZCwgJnNpemUsIHNpemVvZihpbnQp
KSAhPSBzaXplb2YoaW50KSkgew0KICAgICAgICBwZXJyb3IoIndyaXRlIik7
DQogICAgICAgIGV4aXQoMSk7DQogICAgfQ0KICAgIHRvdGFsID0gMDsNCiAg
ICBzdGFydF90aW1lcigpOw0KICAgIGlmKHNpemUgPD0gMCkgeyAvKiBmcm9t
IHNlcnZlciB0byBjbGllbnQgKi8NCiAgICAgICAgc2l6ZSA9IC1zaXplOw0K
ICAgICAgICB3aGlsZSgobiA9IHJlYWQoZmQsIGJ1ZiwgYnVmX3NpemUpKSA+
IDApDQogICAgICAgICAgICB0b3RhbCArPSBuOw0KICAgICAgICB0aW1lID0g
c3RvcF90aW1lcigpOw0KICAgICAgICBmcHJpbnRmKHN0ZGVyciwgIlJlY2Vp
dmVkIik7IA0KICAgIH0NCiAgICBlbHNlIHsgLyogZnJvbSBjbGllbnQgdG8g
c2VydmVyICovDQogICAgICAgIHdoaWxlKHRvdGFsIDwgc2l6ZSkgew0KICAg
ICAgICAgICAgbiA9ICh0b3RhbCArIGJ1Zl9zaXplID4gc2l6ZSkgPyBzaXpl
IC0gdG90YWwgOiBidWZfc2l6ZTsNCiAgICAgICAgICAgIGlmKHdyaXRlKGZk
LCBidWYsIG4pICE9IG4pIHsNCiAgICAgICAgICAgICAgICBwZXJyb3IoIndy
aXRlIik7DQogICAgICAgICAgICAgICAgZXhpdCgxKTsNCiAgICAgICAgICAg
IH0NCiAgICAgICAgICAgIHRvdGFsICs9IG47DQogICAgICAgIH0NCiAgICAg
ICAgdGltZSA9IHN0b3BfdGltZXIoKTsNCiAgICAgICAgZnByaW50ZihzdGRl
cnIsICJTZW50Iik7IA0KICAgIH0NCiAgICBjbG9zZShmZCk7DQogICAgZnBy
aW50ZihzdGRlcnIsICIgJWQgYnl0ZXMgaW4gJS42ZiBzZWNvbmRzXG4iLCB0
b3RhbCwgdGltZSk7DQogICAgZnByaW50ZihzdGRlcnIsICIoYnVmZmVyIHNp
emU9JWQgYnl0ZXMsIHRocm91Z2hwdXQ9JS42ZiBNYnBzKVxuIiwNCiAgICAg
ICAgICAgYnVmX3NpemUsIHRvdGFsKjgvdGltZS8xMDQ4NTc2KTsNCn0NCg0K
bWFpbihpbnQgYXJnYywgY2hhciAqYXJndltdKQ0Kew0KICAgIHN3aXRjaChh
cmdjKSB7DQogICAgICAgIGNhc2UgMjogc2VydmVyKGF0b2koYXJndlsxXSkp
Ow0KICAgICAgICAgICAgICAgIGJyZWFrOw0KICAgICAgICBjYXNlIDU6IGNs
aWVudChhcmd2WzFdLCBhdG9pKGFyZ3ZbMl0pLCBhdG9pKGFyZ3ZbM10pLCBh
dG9pKGFyZ3ZbNF0pKTsNCiAgICAgICAgICAgICAgICBicmVhazsNCiAgICAg
ICAgZGVmYXVsdDogZnByaW50ZihzdGRlcnIsICJVc2FnZTogJXMgPHBvcnQ+
XG4iLCBhcmd2WzBdKTsNCiAgICAgICAgICAgICAgICAgZnByaW50ZihzdGRl
cnIsICJVc2FnZTogJXMgPHNlcnZlcj4gPHBvcnQ+IDxidWZmZXI+IDxkYXRh
IHNpemU+XG4iLA0KICAgICAgICAgICAgICAgICAgICAgICAgIGFyZ3ZbMF0p
Ow0KICAgICAgICAgICAgICAgICBleGl0KDEpOw0KICAgIH0NCn0NCg==
---559023410-851401618-891276400=:28712--