This queue is for tickets about the Math-Symbolic CPAN distribution.

Report information
The Basics
Id:
55842
Status:
open
Priority:
Low/Low
Queue:

People
Owner:
Nobody in particular
Requestors:
pub [...] johnrefior.com
Cc:
AdminCc:

BugTracker
Severity:
Wishlist
Broken in:
0.603
Fixed in:
(no value)



When I ask Math::Symbolic::Derivative to take the partial derivative of A*(x - x_0)^2 + y_0 with respect to x_0, and simplify, it returns the expression (-2A*(x - x_0)^2)/(x - x_0) When I do the same by hand, I get -2A*(x - x_0) The two are equivalent except when x == x_0. Then Symbolic expression returns undef, while mine returns 0. This is causing an error in Algorithm::CurveFit when fitting to a parabola with the above formula, when the x-value of the initial guess of the vertex is equal to one of the x-value data points submitted to the curve fitting subroutine (x == x_0). Please see the attached file for a test script that will reproduce the error. Running perl v5.8.8 built for i686-linux, using Math::Symbolic version 0.603, and Algorithm::CurveFit version 1.04. I also intend to submit a bug under Algorithm::CurveFit so that they are aware of the issue and the lack of handling of an undef return from deriv->value(). - John Refior
Subject: math_sym_deriv_test.pl
#!/usr/bin/perl use strict; use warnings; use Test::More tests => 1; use Math::Symbolic; use Data::Dumper; my $formula = 'A*(x-x_0)^2+y_0'; my $raw = Math::Symbolic::Operator->new( 'partial_derivative', $formula, 'x_0' ); my $deriv = $raw->simplify()->apply_derivatives()->simplify(); ######################################################################## # When I do this partial derivative by hand, I get -2 * A * (x - x_0). # # However, $deriv is now (-2A*(x - x_0)^2)/(x - x_0). # # These are equivalent, except when x == x_0. Then mine by hand is 0, # # and $deriv->value is undef. This is causing an error in # # Algorithm::CurveFit for certain cases (when the x-value of the # # initial guess of the vertex is equal to one of the x-data points # # submitted) I've encountered when trying to perform curve fitting on # # a parabola. # print 'Simplified derivative expression is ', Dumper($deriv), "\n\n"; is( $deriv->value('x' => 2, 'A' => -5, 'x_0' => 2, 'y_0' => 20), 0, 'expect 0 when x=2'); print "Tests complete.\n";
Subject: Math::Symbolic derivative/simplification
Hi John, sorry for the delayed reply. What you're experiencing is an artifact of the simplification routines of Math::Symbolic. They're hacky and quite honestly, I wrote them in the summer break after my first semester at college. While I could *probably* do a better job today, I have no means to do so. It's also a generally tricky problem, so issues like the one you reported would just become less likely, not impossible. Therefore, I'm marking the bug report as "wishlist" because it's not really a bug. The derivative is correct. The simplification fails to find the common x-x_0, but it's heuristics. I don't have the time to do much in terms of maintenance, but for posterity's sake, these are the best ideas I have for "fixing" the issue: a) Rewrite the simplification routines. This would also require improving the data structures. Simple-minded trees for all operators are difficult to match against. b) Extend Algorithm::CurveFit in order to allow users to override the buggy behaviour. b1) Either allow passing in subroutines for evaluation of the function and its derivative instead of formulas, b2) or allow the user to explicitly pass in the derivative as a formula. In principle, since you know what the problem is, you could write your own, special-case simplification routine and plug it into Math::Symbolic. For details on how this can work, have a look at http://search.cpan.org/~smueller/Math-Symbolic-Custom-Simplification-1.01/ Such a custom simplification could probably easily be implemented using Math::Symbolic::Custom::Transformation which provides facilities to apply pattern matching/replacement on the Math::Symbolic trees. I'm sorry I can't just fix the underlying problem and hope that this helps at least a bit. Best regards, Steffen
From: pub@johnrefior.com
Steffen, Thank you for your response, and for creating and maintaining these useful modules. I understand that the root problem is tricky and not easily solved. I have implemented a hack in my code, to slightly modify x_0 when one of the x data points is equal to my initial guess for x_0. In testing so far this works without problem. So we have a workaround (slightly ugly hack) that allows us to get by for now without modifying the CPAN modules (our preference so that we can easily upgrade to new module versions). I'm not a mathematician, so I could be wrong, but I'm not sure I quite agree that the derivative here is correct and that the only issue is with the simplification routine. It's my understanding that 0/0 is undefined in mathematics, and that 0/0 != 0. Based on that, ((x-y)^2)/(x-y) cannot be simplified to (x-y) unless you know that x != y, which is not the case here. When x == y, one is 0, the other is undefined, and therefore they are not equivalent expressions. I believe the correct derivative in this case is -2*A*(x-x_0). Thanks again for your work on these modules and your attention to this issue. Regards, John
From: pub@johnrefior.com
Steffen, I wanted to write again to ask a couple questions. I think perhaps you are suggesting that the simplification routine could be modified to simplify ((x - y)^2)/(x - y) to (x - y). This would indeed solve the problem I am running into. Do you think it would introduce any other bugs, e.g. where the method should return undef but would then return 0 instead? In which module and method would you make this modification? Thanks again for your help. Regards, John
Subject: Re: [rt.cpan.org #55842] Simplification deficiency affects Algorithm::CurveFit
Date: Thu, 25 Mar 2010 16:27:11 +0100
To: bug-Math-Symbolic@rt.cpan.org
From: Steffen Mueller <smueller@cpan.org>
Hi John, John Refior via RT wrote:
Show quoted text
> Queue: Math-Symbolic > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=55842 >
Show quoted text
> I wanted to write again to ask a couple questions. I think perhaps > you are suggesting that the simplification routine could be modified > to simplify ((x - y)^2)/(x - y) to (x - y).
Yes. I do realize that this runs into issues with x==y, but the module can hardly know that, so my hunch is the simplification is valid.
Show quoted text
> This would indeed solve the problem I am running into. Do you think > it would introduce any other bugs, e.g. where the method should return > undef but would then return 0 instead?
No, I don't *think* so.
Show quoted text
> In which module and method would you make this modification?
The simplification code is in Math::Symbolic::Operator. Beware, hic sunt dracones. I was actually thinking of a slightly different approach. The Math::Symbolic::Custom::Simplifcation module (IIRC) implements a sort of pluggable simplification so you can write your own entirely. If it's possible to call the default simplification code from your custom one, you can simply do an extra step after the normal simplification had a go. Then you could try to do something like this (pseudo-code): (->old_simplify() is pseudo code for 'figure out how to call the normal simplfication routine') use Math::Symbolic::Custom::Transformation 'new_trafo'; my $replacer = new_trafo('x-x_0' => 'mysecretvariable'); sub my_simplify { my $f = shift; my $simpler = $f->old_simplify(); $simpler = $replacer->apply($simpler) ->old_simplify() ->implement('mysecretvariable' => 'x - x_0'); return $simpler; } I realize that this is likely a lot of trouble. If you added the option to Algorithm::CurveFit to accept a derivative formula instead of computing it by itself, then you could just do this manually without overriding any simplification routines. Best regards, Steffen
From: pub@johnrefior.com
Steffen, Thank you for your prompt and thorough reply. I've had to move on to other work and I haven't yet had time to investigate all these options with the thoroughness they deserve. I hope to come back to it in the future when I have some more free time. Meanwhile, my hack to our code seems to be working. Thanks again, John
Hi John, On Wed Apr 07 10:17:20 2010, jrefior wrote:
Show quoted text
> Thank you for your prompt and thorough reply. I've had to move on to > other work and I haven't yet had time to investigate all these options > with the thoroughness they deserve. > > I hope to come back to it in the future when I have some more free time. > Meanwhile, my hack to our code seems to be working.
let me add a late note. I recently added a numeric derivative (five point stencil) fallback to Algorithm::CurveFit that should catch some of these case where the symbolic derivation fails. This doesn't resolve the underlying simplification problem, of course. Best regards, Steffen


This service runs on Request Tracker, is sponsored by The Perl Foundation, and maintained by Best Practical Solutions.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.