Weekly benchmark results [1/1st week]

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Weekly benchmark results [1/1st week]

yo.fujita
Hi, all

I will inform of this week's benchmark result.

The tool used now is as follows.
 - unixbench4.1.0
 - bonnie++-1.03
 - ltp-full-20051103
 - iozone3_191
 - lmbench-3.0-a5

TEST ENVIRONMENT
    Machine          : Tiger4
    KERN             : 2.6.12-xenU
    changeset        : 8474:22f28fffa0a8 (1/5 latest)
    Dom0 OS          : RHEL4 U2 (no SMP)
    DomU OS          : RHEL4 U2 (no SMP)
    No. of DomU's    : 1

SUMMARY:
     - changeset8474, All benchmarks normally ended
issues:
     -

TEST RESULT
    unixbench4.1.0    : PASS
    bonnie++-1.03    : PASS
    ltp-full-20051103 : 52/816 FAIL (Please see the attached files)
    iozone3_191       : PASS
    lmbench-3.0-a5    : PASS

Thanks and best regards,
Fujita and Fujisu members

_______________________________________________
Xen-ia64-devel mailing list
[hidden email]
http://lists.xensource.com/xen-ia64-devel

ltp-domU-20060107.txt (41K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: Weekly benchmark results [1/1st week]

Dan Magenheimer
Hi Fujita-san --

Thanks very much for running these tests weekly!

On LTP, a few questions:
- Are the failures reproducible?  Or are some failures
  random?  (may be the same problem as the gcc failures)
- Of the 52 failures, I think some of the tests also fail
  on Linux/ia64.  We should probably exclude those (or
  not count them as Xen failures).
- I saw much fewer failures when I tried LTP a couple of
  weeks ago.  Many tests failed for me at first because
  there was not enough disk space.  Do you have enough
  disk space on the domU filesystem?

Thanks,
Dan

> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]] On Behalf
> Of yo.fujita
> Sent: Saturday, January 07, 2006 1:31 AM
> To: [hidden email]
> Subject: [Xen-ia64-devel] Weekly benchmark results [1/1st week]
>
> Hi, all
>
> I will inform of this week's benchmark result.
>
> The tool used now is as follows.
>  - unixbench4.1.0
>  - bonnie++-1.03
>  - ltp-full-20051103
>  - iozone3_191
>  - lmbench-3.0-a5
>
> TEST ENVIRONMENT
>     Machine          : Tiger4
>     KERN             : 2.6.12-xenU
>     changeset        : 8474:22f28fffa0a8 (1/5 latest)
>     Dom0 OS          : RHEL4 U2 (no SMP)
>     DomU OS          : RHEL4 U2 (no SMP)
>     No. of DomU's    : 1
>
> SUMMARY:
>      - changeset8474, All benchmarks normally ended
> issues:
>      -
>
> TEST RESULT
>     unixbench4.1.0    : PASS
>     bonnie++-1.03    : PASS
>     ltp-full-20051103 : 52/816 FAIL (Please see the attached files)
>     iozone3_191       : PASS
>     lmbench-3.0-a5    : PASS
>
> Thanks and best regards,
> Fujita and Fujisu members
>

_______________________________________________
Xen-ia64-devel mailing list
[hidden email]
http://lists.xensource.com/xen-ia64-devel
Reply | Threaded
Open this post in threaded view
|

RE: Weekly benchmark results [1/1st week]

yo.fujita
Hi, Dan-san

Sorry for a late mail.
I retried LTP.
As a result, it became an almost similar result.
        Total Failures: 52 (Please look at the attached file details.)

The difference is only the following compared with the last result.
        762,763c762,763
        < gf16                           FAIL       1    
        < gf17                           PASS       0    
        ---
        > gf16                           PASS       0    
        > gf17                           FAIL       1    

> - I saw much fewer failures when I tried LTP a couple of
>   weeks ago.  Many tests failed for me at first because
>   there was not enough disk space.  Do you have enough
>   disk space on the domU filesystem?

Yes, Some of the error were the one due to disk space shortage.
After increasing the disk space, I will try again.

Thanks
Fujita

> -----Original Message-----
> From: Magenheimer, Dan (HP Labs Fort Collins)
[mailto:[hidden email]]

> Sent: Monday, January 09, 2006 9:00 AM
> To: yo.fujita; [hidden email]
> Subject: RE: [Xen-ia64-devel] Weekly benchmark results [1/1st week]
>
> Hi Fujita-san --
>
> Thanks very much for running these tests weekly!
>
> On LTP, a few questions:
> - Are the failures reproducible?  Or are some failures
>   random?  (may be the same problem as the gcc failures)
> - Of the 52 failures, I think some of the tests also fail
>   on Linux/ia64.  We should probably exclude those (or
>   not count them as Xen failures).
> - I saw much fewer failures when I tried LTP a couple of
>   weeks ago.  Many tests failed for me at first because
>   there was not enough disk space.  Do you have enough
>   disk space on the domU filesystem?
>
> Thanks,
> Dan
>
> > -----Original Message-----
> > From: [hidden email]
> > [mailto:[hidden email]] On Behalf
> > Of yo.fujita
> > Sent: Saturday, January 07, 2006 1:31 AM
> > To: [hidden email]
> > Subject: [Xen-ia64-devel] Weekly benchmark results [1/1st week]
> >
> > Hi, all
> >
> > I will inform of this week's benchmark result.
> >
> > The tool used now is as follows.
> >  - unixbench4.1.0
> >  - bonnie++-1.03
> >  - ltp-full-20051103
> >  - iozone3_191
> >  - lmbench-3.0-a5
> >
> > TEST ENVIRONMENT
> >     Machine          : Tiger4
> >     KERN             : 2.6.12-xenU
> >     changeset        : 8474:22f28fffa0a8 (1/5 latest)
> >     Dom0 OS          : RHEL4 U2 (no SMP)
> >     DomU OS          : RHEL4 U2 (no SMP)
> >     No. of DomU's    : 1
> >
> > SUMMARY:
> >      - changeset8474, All benchmarks normally ended
> > issues:
> >      -
> >
> > TEST RESULT
> >     unixbench4.1.0    : PASS
> >     bonnie++-1.03    : PASS
> >     ltp-full-20051103 : 52/816 FAIL (Please see the attached files)
> >     iozone3_191       : PASS
> >     lmbench-3.0-a5    : PASS
> >
> > Thanks and best regards,
> > Fujita and Fujisu members
> >

_______________________________________________
Xen-ia64-devel mailing list
[hidden email]
http://lists.xensource.com/xen-ia64-devel

ltp-domU20060111.txt (39K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: Weekly benchmark results [1/1st week]

yo.fujita
Hi, Dan-san

I changed DomU's disk image size 4G -> 7G. And, LTP was executed.
As a result, it did not become the result of the expectation.
        Total Failures: 49 (Please look at the attached file details.)

Diff:
        < gf15                           FAIL       1    
        ---
        > gf15                           PASS       0    
        706,707c706,707
        < gf17                           FAIL       1    
        < gf18                           FAIL       1    
        ---
        > gf17                           PASS       0    
        > gf18                           PASS       0    
        824c824

How many Fail did you have on LTP-domU?
And, how many is the size of the image file?

I tried LTP on native linux, it's Total Failures: 0

Thanks
Fujita

> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]] On Behalf Of yo.fujita
> Sent: Wednesday, January 11, 2006 12:07 PM
> To: 'Magenheimer, Dan (HP Labs Fort Collins)';
> [hidden email]
> Subject: RE: [Xen-ia64-devel] Weekly benchmark results [1/1st week]
>
> Hi, Dan-san
>
> Sorry for a late mail.
> I retried LTP.
> As a result, it became an almost similar result.
> Total Failures: 52 (Please look at the attached file details.)
>
> The difference is only the following compared with the last result.
> 762,763c762,763
> < gf16                           FAIL       1
> < gf17                           PASS       0
> ---
> > gf16                           PASS       0
> > gf17                           FAIL       1
>
> > - I saw much fewer failures when I tried LTP a couple of
> >   weeks ago.  Many tests failed for me at first because
> >   there was not enough disk space.  Do you have enough
> >   disk space on the domU filesystem?
>
> Yes, Some of the error were the one due to disk space shortage.
> After increasing the disk space, I will try again.
>
> Thanks
> Fujita
>
> > -----Original Message-----
> > From: Magenheimer, Dan (HP Labs Fort Collins)
> [mailto:[hidden email]]
> > Sent: Monday, January 09, 2006 9:00 AM
> > To: yo.fujita; [hidden email]
> > Subject: RE: [Xen-ia64-devel] Weekly benchmark results [1/1st week]
> >
> > Hi Fujita-san --
> >
> > Thanks very much for running these tests weekly!
> >
> > On LTP, a few questions:
> > - Are the failures reproducible?  Or are some failures
> >   random?  (may be the same problem as the gcc failures)
> > - Of the 52 failures, I think some of the tests also fail
> >   on Linux/ia64.  We should probably exclude those (or
> >   not count them as Xen failures).
> > - I saw much fewer failures when I tried LTP a couple of
> >   weeks ago.  Many tests failed for me at first because
> >   there was not enough disk space.  Do you have enough
> >   disk space on the domU filesystem?
> >
> > Thanks,
> > Dan
> >
> > > -----Original Message-----
> > > From: [hidden email]
> > > [mailto:[hidden email]] On Behalf
> > > Of yo.fujita
> > > Sent: Saturday, January 07, 2006 1:31 AM
> > > To: [hidden email]
> > > Subject: [Xen-ia64-devel] Weekly benchmark results [1/1st week]
> > >
> > > Hi, all
> > >
> > > I will inform of this week's benchmark result.
> > >
> > > The tool used now is as follows.
> > >  - unixbench4.1.0
> > >  - bonnie++-1.03
> > >  - ltp-full-20051103
> > >  - iozone3_191
> > >  - lmbench-3.0-a5
> > >
> > > TEST ENVIRONMENT
> > >     Machine          : Tiger4
> > >     KERN             : 2.6.12-xenU
> > >     changeset        : 8474:22f28fffa0a8 (1/5 latest)
> > >     Dom0 OS          : RHEL4 U2 (no SMP)
> > >     DomU OS          : RHEL4 U2 (no SMP)
> > >     No. of DomU's    : 1
> > >
> > > SUMMARY:
> > >      - changeset8474, All benchmarks normally ended
> > > issues:
> > >      -
> > >
> > > TEST RESULT
> > >     unixbench4.1.0    : PASS
> > >     bonnie++-1.03    : PASS
> > >     ltp-full-20051103 : 52/816 FAIL (Please see the attached files)
> > >     iozone3_191       : PASS
> > >     lmbench-3.0-a5    : PASS
> > >
> > > Thanks and best regards,
> > > Fujita and Fujisu members
> > >

_______________________________________________
Xen-ia64-devel mailing list
[hidden email]
http://lists.xensource.com/xen-ia64-devel

ltp-domU20060111-2.txt (39K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: Weekly benchmark results [1/1st week]

Dan Magenheimer
In reply to this post by yo.fujita
Is there a script/program in the ltp directories that
outputs the pass/fail file you are sending?  I have
just been grep'ing the output of ./runalltests.sh for
"FAIL".  On my last run (domU on an rx2600):

"grep FAIL ltp.out | wc" is 23
"grep FAIL ltp.out | grep -v pow_out | wc" is 3

The three FAIL lines are:

sig_rev     1  FAIL  :  alarm() fails to send SIGALRM to execde process
float_power    1  FAIL  :  Test failed
INFO: pan reported some tests FAIL

Dan

> -----Original Message-----
> From: yo.fujita [mailto:[hidden email]]
> Sent: Wednesday, January 11, 2006 1:05 AM
> To: Magenheimer, Dan (HP Labs Fort Collins);
> [hidden email]
> Subject: RE: [Xen-ia64-devel] Weekly benchmark results [1/1st week]
>
> Hi, Dan-san
>
> I changed DomU's disk image size 4G -> 7G. And, LTP was executed.
> As a result, it did not become the result of the expectation.
> Total Failures: 49 (Please look at the attached file details.)
>
> Diff:
> < gf15                           FAIL       1    
> ---
> > gf15                           PASS       0    
> 706,707c706,707
> < gf17                           FAIL       1    
> < gf18                           FAIL       1    
> ---
> > gf17                           PASS       0    
> > gf18                           PASS       0    
> 824c824
>
> How many Fail did you have on LTP-domU?
> And, how many is the size of the image file?
>
> I tried LTP on native linux, it's Total Failures: 0
>
> Thanks
> Fujita
>
> > -----Original Message-----
> > From: [hidden email]
> > [mailto:[hidden email]] On
> Behalf Of yo.fujita
> > Sent: Wednesday, January 11, 2006 12:07 PM
> > To: 'Magenheimer, Dan (HP Labs Fort Collins)';
> > [hidden email]
> > Subject: RE: [Xen-ia64-devel] Weekly benchmark results [1/1st week]
> >
> > Hi, Dan-san
> >
> > Sorry for a late mail.
> > I retried LTP.
> > As a result, it became an almost similar result.
> > Total Failures: 52 (Please look at the attached file details.)
> >
> > The difference is only the following compared with the last result.
> > 762,763c762,763
> > < gf16                           FAIL       1
> > < gf17                           PASS       0
> > ---
> > > gf16                           PASS       0
> > > gf17                           FAIL       1
> >
> > > - I saw much fewer failures when I tried LTP a couple of
> > >   weeks ago.  Many tests failed for me at first because
> > >   there was not enough disk space.  Do you have enough
> > >   disk space on the domU filesystem?
> >
> > Yes, Some of the error were the one due to disk space shortage.
> > After increasing the disk space, I will try again.
> >
> > Thanks
> > Fujita
> >
> > > -----Original Message-----
> > > From: Magenheimer, Dan (HP Labs Fort Collins)
> > [mailto:[hidden email]]
> > > Sent: Monday, January 09, 2006 9:00 AM
> > > To: yo.fujita; [hidden email]
> > > Subject: RE: [Xen-ia64-devel] Weekly benchmark results
> [1/1st week]
> > >
> > > Hi Fujita-san --
> > >
> > > Thanks very much for running these tests weekly!
> > >
> > > On LTP, a few questions:
> > > - Are the failures reproducible?  Or are some failures
> > >   random?  (may be the same problem as the gcc failures)
> > > - Of the 52 failures, I think some of the tests also fail
> > >   on Linux/ia64.  We should probably exclude those (or
> > >   not count them as Xen failures).
> > > - I saw much fewer failures when I tried LTP a couple of
> > >   weeks ago.  Many tests failed for me at first because
> > >   there was not enough disk space.  Do you have enough
> > >   disk space on the domU filesystem?
> > >
> > > Thanks,
> > > Dan
> > >
> > > > -----Original Message-----
> > > > From: [hidden email]
> > > > [mailto:[hidden email]] On Behalf
> > > > Of yo.fujita
> > > > Sent: Saturday, January 07, 2006 1:31 AM
> > > > To: [hidden email]
> > > > Subject: [Xen-ia64-devel] Weekly benchmark results [1/1st week]
> > > >
> > > > Hi, all
> > > >
> > > > I will inform of this week's benchmark result.
> > > >
> > > > The tool used now is as follows.
> > > >  - unixbench4.1.0
> > > >  - bonnie++-1.03
> > > >  - ltp-full-20051103
> > > >  - iozone3_191
> > > >  - lmbench-3.0-a5
> > > >
> > > > TEST ENVIRONMENT
> > > >     Machine          : Tiger4
> > > >     KERN             : 2.6.12-xenU
> > > >     changeset        : 8474:22f28fffa0a8 (1/5 latest)
> > > >     Dom0 OS          : RHEL4 U2 (no SMP)
> > > >     DomU OS          : RHEL4 U2 (no SMP)
> > > >     No. of DomU's    : 1
> > > >
> > > > SUMMARY:
> > > >      - changeset8474, All benchmarks normally ended
> > > > issues:
> > > >      -
> > > >
> > > > TEST RESULT
> > > >     unixbench4.1.0    : PASS
> > > >     bonnie++-1.03    : PASS
> > > >     ltp-full-20051103 : 52/816 FAIL (Please see the
> attached files)
> > > >     iozone3_191       : PASS
> > > >     lmbench-3.0-a5    : PASS
> > > >
> > > > Thanks and best regards,
> > > > Fujita and Fujisu members
> > > >
>

_______________________________________________
Xen-ia64-devel mailing list
[hidden email]
http://lists.xensource.com/xen-ia64-devel
Reply | Threaded
Open this post in threaded view
|

RE: Weekly benchmark results [1/1st week]

yo.fujita
Hi, Dan

My sending file is results log.

#./runalltests.sh -p -l /tools/results/ltp/ltp-domU20060111

I did greping to the output file by the same procedure as you. (
ltp-domU-execlog-20060111.log is output log )

"grep FAIL ltp-domU-execlog-20060111.log | wc" is 23
"grep FAIL ltp-domU-execlog-20060111.log | grep -v pow_out | wc" is 3

sig_rev     1  FAIL  :  alarm() fails to send SIGALRM to execde process
float_power    1  FAIL  :  Test failed
INFO: pan reported some tests FAIL

The same as Dan's resulted.

Thanks
Fujita

> -----Original Message-----
> From: Magenheimer, Dan (HP Labs Fort Collins)
[mailto:[hidden email]]

> Sent: Wednesday, January 11, 2006 11:31 PM
> To: yo.fujita; [hidden email]
> Subject: RE: [Xen-ia64-devel] Weekly benchmark results [1/1st week]
>
> Is there a script/program in the ltp directories that
> outputs the pass/fail file you are sending?  I have
> just been grep'ing the output of ./runalltests.sh for
> "FAIL".  On my last run (domU on an rx2600):
>
> "grep FAIL ltp.out | wc" is 23
> "grep FAIL ltp.out | grep -v pow_out | wc" is 3
>
> The three FAIL lines are:
>
> sig_rev     1  FAIL  :  alarm() fails to send SIGALRM to execde process
> float_power    1  FAIL  :  Test failed
> INFO: pan reported some tests FAIL
>
> Dan
>
> > -----Original Message-----
> > From: yo.fujita [mailto:[hidden email]]
> > Sent: Wednesday, January 11, 2006 1:05 AM
> > To: Magenheimer, Dan (HP Labs Fort Collins);
> > [hidden email]
> > Subject: RE: [Xen-ia64-devel] Weekly benchmark results [1/1st week]
> >
> > Hi, Dan-san
> >
> > I changed DomU's disk image size 4G -> 7G. And, LTP was executed.
> > As a result, it did not become the result of the expectation.
> > Total Failures: 49 (Please look at the attached file details.)
> >
> > Diff:
> > < gf15                           FAIL       1
> > ---
> > > gf15                           PASS       0
> > 706,707c706,707
> > < gf17                           FAIL       1
> > < gf18                           FAIL       1
> > ---
> > > gf17                           PASS       0
> > > gf18                           PASS       0
> > 824c824
> >
> > How many Fail did you have on LTP-domU?
> > And, how many is the size of the image file?
> >
> > I tried LTP on native linux, it's Total Failures: 0
> >
> > Thanks
> > Fujita
> >
> > > -----Original Message-----
> > > From: [hidden email]
> > > [mailto:[hidden email]] On
> > Behalf Of yo.fujita
> > > Sent: Wednesday, January 11, 2006 12:07 PM
> > > To: 'Magenheimer, Dan (HP Labs Fort Collins)';
> > > [hidden email]
> > > Subject: RE: [Xen-ia64-devel] Weekly benchmark results [1/1st week]
> > >
> > > Hi, Dan-san
> > >
> > > Sorry for a late mail.
> > > I retried LTP.
> > > As a result, it became an almost similar result.
> > > Total Failures: 52 (Please look at the attached file details.)
> > >
> > > The difference is only the following compared with the last result.
> > > 762,763c762,763
> > > < gf16                           FAIL       1
> > > < gf17                           PASS       0
> > > ---
> > > > gf16                           PASS       0
> > > > gf17                           FAIL       1
> > >
> > > > - I saw much fewer failures when I tried LTP a couple of
> > > >   weeks ago.  Many tests failed for me at first because
> > > >   there was not enough disk space.  Do you have enough
> > > >   disk space on the domU filesystem?
> > >
> > > Yes, Some of the error were the one due to disk space shortage.
> > > After increasing the disk space, I will try again.
> > >
> > > Thanks
> > > Fujita
> > >
> > > > -----Original Message-----
> > > > From: Magenheimer, Dan (HP Labs Fort Collins)
> > > [mailto:[hidden email]]
> > > > Sent: Monday, January 09, 2006 9:00 AM
> > > > To: yo.fujita; [hidden email]
> > > > Subject: RE: [Xen-ia64-devel] Weekly benchmark results
> > [1/1st week]
> > > >
> > > > Hi Fujita-san --
> > > >
> > > > Thanks very much for running these tests weekly!
> > > >
> > > > On LTP, a few questions:
> > > > - Are the failures reproducible?  Or are some failures
> > > >   random?  (may be the same problem as the gcc failures)
> > > > - Of the 52 failures, I think some of the tests also fail
> > > >   on Linux/ia64.  We should probably exclude those (or
> > > >   not count them as Xen failures).
> > > > - I saw much fewer failures when I tried LTP a couple of
> > > >   weeks ago.  Many tests failed for me at first because
> > > >   there was not enough disk space.  Do you have enough
> > > >   disk space on the domU filesystem?
> > > >
> > > > Thanks,
> > > > Dan
> > > >
> > > > > -----Original Message-----
> > > > > From: [hidden email]
> > > > > [mailto:[hidden email]] On Behalf
> > > > > Of yo.fujita
> > > > > Sent: Saturday, January 07, 2006 1:31 AM
> > > > > To: [hidden email]
> > > > > Subject: [Xen-ia64-devel] Weekly benchmark results [1/1st week]
> > > > >
> > > > > Hi, all
> > > > >
> > > > > I will inform of this week's benchmark result.
> > > > >
> > > > > The tool used now is as follows.
> > > > >  - unixbench4.1.0
> > > > >  - bonnie++-1.03
> > > > >  - ltp-full-20051103
> > > > >  - iozone3_191
> > > > >  - lmbench-3.0-a5
> > > > >
> > > > > TEST ENVIRONMENT
> > > > >     Machine          : Tiger4
> > > > >     KERN             : 2.6.12-xenU
> > > > >     changeset        : 8474:22f28fffa0a8 (1/5 latest)
> > > > >     Dom0 OS          : RHEL4 U2 (no SMP)
> > > > >     DomU OS          : RHEL4 U2 (no SMP)
> > > > >     No. of DomU's    : 1
> > > > >
> > > > > SUMMARY:
> > > > >      - changeset8474, All benchmarks normally ended
> > > > > issues:
> > > > >      -
> > > > >
> > > > > TEST RESULT
> > > > >     unixbench4.1.0    : PASS
> > > > >     bonnie++-1.03    : PASS
> > > > >     ltp-full-20051103 : 52/816 FAIL (Please see the
> > attached files)
> > > > >     iozone3_191       : PASS
> > > > >     lmbench-3.0-a5    : PASS
> > > > >
> > > > > Thanks and best regards,
> > > > > Fujita and Fujisu members
> > > > >
> >



_______________________________________________
Xen-ia64-devel mailing list
[hidden email]
http://lists.xensource.com/xen-ia64-devel