Review of the past two weeks

In the last two weeks, the main work focused on developing new solvers for different kinds of BVPs, improving the testing coverage of the implemented solvers, and fixing a lot of possible bugs, some progress can be summarized as follows:

Implemented more BVDAE testing problems from BVDAE-oriented research papers, linear and nonlinear, index-1 and index-2 problems, during the comparing process, I found there are some differences from the output of COLDAE and my implementation, mainly about the collocation system setup and solving part, which is the most important part of the solver, so I need to put more time on polishing this.
Implemented MIRKN methods for SecondOrderBVProblem, the main nonlinear system-solving process is done, and are getting a correct result now, but there are still some issues with the sparsity detection and automatic differentiation stuff that need to be dug in.

For example, as for second-order boundary value problem example:

with boundary conditions:

To utilize the MIRKN method to solve this second-order system with BoundaryValueDiffEq.jl, we only need to follow what we do when solving first-order BVP system with MIRK methods:

function test(ddu, du, u, p, t)
    ϵ = 0.1
    ddu[1] = u[2]
    ddu[2] = (-u[1]*du[2] - u[3]*du[3])/ϵ
    ddu[3] = (du[1]*u[3] - u[1]*du[3])/ϵ
end

function bc(res, du, u, p, t)
    res[1] = u[1][1]
    res[2] = u[end][1]
    res[3] = u[1][3] + 1
    res[4] = u[end][3] - 1
    res[5] = du[1][1]
    res[6] = du[end][1]
end

u0 = [1.0, 1.0, 1.0]
tspan = (0.0, 1.0)

prob = SecondOrderBVProblem(test, bc, u0, tspan)
sol=solve(prob, MIRKN4(), dt=0.01)

The solution from BoundaryValueDiffEq.jl would be:

More TODOs:

Fix the sparsity detection and automatic differentiation issues and send the current implementations of MIRKN as soon as possible.
According to the different example problems find possible bugs in the COLDAE implementation and see if there are any improvements we need to complete.
Continue to refactor the solver, especially the collocation equations setting up part.