Results

Important note: A number of problems in this test suite have no antiderivative in closed form. This means the antiderivative of these integrals can not be expressed in terms of elementary, special functions or Hypergeometric2F1 functions. RootSum and RootOf are not allowed.

If a CAS returns the above integral unevaluated within the time limit, then the result is counted as passed and assigned an A grade.

However, if CAS times out, then it is assigned an F grade even if the integral is not integrable, as this implies CAS could not determine that the integral is not integrable in the time limit.

If a CAS returns an antiderivative to such an integral, it is assigned an A grade automatically and this special result is listed in the introduction section of each individual test report to make it easy to identify as this can be important result to investigate.


System	% solved	% Failed

Mathematica	96.65 ( 21864 )	3.35 ( 758 )
Rubi	96.16 ( 21754 )	3.84 ( 868 )
Fricas	93.75 ( 21207 )	6.25 ( 1415 )
Maple	91.89 ( 20788 )	8.11 ( 1834 )
IntegrateAlgebraic	61.28 ( 13863 )	38.72 ( 8759 )
Giac	77.45 ( 17520 )	22.55 ( 5102 )
Mupad	70.46 ( 15939 )	29.54 ( 6683 )
Maxima	68.31 ( 15452 )	31.69 ( 7170 )
Sympy	54.15 ( 12250 )	45.85 ( 10372 )

Table 1.1:Percentage solved for each CAS

The table below gives additional break down of the grading of quality of the antiderivatives generated by each CAS. The grading is given using the letters A,B,C and F with A being the best quality. The grading is accomplished by comparing the antiderivative generated with the optimal antiderivatives included in the test suite. The following table describes the meaning of these grades.


grade	description


A	Integral was solved and antiderivative is optimal in quality and leaf size.

B	Integral was solved and antiderivative is optimal in quality but leaf size is larger than twice the optimal antiderivatives leaf size.

C	Integral was solved and antiderivative is non-optimal in quality. This can be due to one or more of the following reasons antiderivative contains a hypergeometric function and the optimal antiderivative does not. antiderivative contains a special function and the optimal antiderivative does not. antiderivative contains the imaginary unit and the optimal antiderivative does not.

F	Integral was not solved. Either the integral was returned unevaluated within the time limit, or it timed out, or CAS hanged or crashed or an exception was raised.

Table 1.2:Description of grading applied to integration result

Grading is implemented for all CAS systems in this version except for CAS Mupad where a grade of B is automatically assigned as a place holder for all integrals it completes on time.


System	% A grade	% B grade	% C grade	% F grade


Rubi	93.93	0.41	1.82	3.84

Mathematica	78.86	2.79	14.98	3.35

IntegrateAlgebraic	55.37	3.64	2.27	38.72

Fricas	69.12	24.12	0.5	6.25

Maple	63.41	20.08	8.41	8.11

Maxima	59.26	8.6	0.45	31.69

Giac	58.57	18.73	0.15	22.55

Sympy	39.29	10.82	4.04	45.85

Mupad	N/A	70.46	0.	29.54

Table 1.3:Antiderivative Grade distribution for each CAS

1.2.1 Time and leaf size Performance

The table below summarizes the performance of each CAS system in terms of time used and leaf size of results.


System	Mean time (sec)	Mean size	Normalized mean	Median size	Normalized median

Rubi	0.25	136.14	1.1	94.	1.
Mathematica	0.3	219.07	2.1	74.	0.92
Maple	0.34	883.32	4.8	92.	1.06
Maxima	1.31	140.62	1.32	76.	0.98
Fricas	2.42	410.66	2.46	120.	1.4
Sympy	9.7	259.23	2.75	83.	1.2
Giac	0.98	285.73	1.83	97.	1.06
Mupad	1.86	880.96	3.34	77.	0.98
IntegrateAlgebraic	2.84	246.87	1.54	97.	1.

Table 1.4:Time and leaf size performance for each CAS

1.2 Results

1.2.1 Time and leaf size Performance