Subject: CVS commit: pkgsrc/lang/py-uncompyle6
From: Adam Ciarcinski
Date: 2020-02-13 20:27:36
Message id: 20200213192736.45027FBF4@cvs.NetBSD.org

Log Message:
py-uncompyle6: updated to 3.6.4

3.6.4:
plateau

The main focus in this release was fix some of the more glaring problems creapt \ 
in from the last release due to that refactor.

uncompyle6 code is at a plateau where what is most needed is a code refactoring. \ 
In doing this, until everything refactored and replaced, decomplation may get \ 
worse.

Therefore, this release largely serves as a checkpoint before more major upheaval.

The upheaval, in started last release, I believe the pinnicle was around c90ff51 \ 
which wasn't a release. I suppose I should tag that.

After c90ff5, I started down the road of redoing control flow in a more \ 
comprehensible, debuggable, and scalable way. See The Control Flow Mess.

The bulk of the refactoring going on in the decompyle3 project, but I try to \ 
trickle down the changes.

It is tricky because the changes are large and I have to figure decompose things \ 
so that little testable pieces can be done. And there is also the problem that \ 
what is in decompyle3 is incomplete as well.

Other than control flow, another change that will probably happen in the next \ 
release is to redo the grammar for lambda expressions. Right now, we treat them \ 
as Python statements, you know, things with compound statements in them. But \ 
lambdas aren't that. And so there is hackery to paper over difference making a \ 
statement out of an expression the wrong thing to do. For example, a return of \ 
an "and" expression can be expressed as nested "if" \ 
statements with return inside them, but the "if" variant of the \ 
bytecode is not valid in a lambda.

In the decompyle3 code, I've gone down the road making the grammar goal symbol \ 
be an expression. This also offers the opportunity to split the grammar making \ 
parsing inside lambda not only more reliable because the wrong choices don't \ 
exist, but also simpler and faster because all those rules just need don't need \ 
to exist in parsing.

I cringe in thinking about how the code has lived for so long without noticing \ 
such a simple stupidity, and lapse of sufficient thought.

3.6.3:
Martin and Susanne

Of late, every release fixes major gaps and embarrassments of the last release....

And in some cases, like this one, exposes lacuna and rot.

I now have [control] flow under control, even if it isn't the most optimal way.

I now have greatly expanded automated testing.

On the most recent Python versions I regularly decompile thousands of Python \ 
programs that are distributed with Python. when it is possible, I then decompile \ 
Python's standard test suite distributed with Python and run the decompiled \ 
source code which basically checks itself. This amounts to about 250 test \ 
programs per version. This is in addition to the 3 CI testing services which do \ 
different things.

Does this mean the decompiler works perfectly? No. There are still a dozen or so \ 
failing programs, although the actual number of bugs is probably smaller though.

However, in perparation of a more major refactoring of the parser grammar, this \ 
release was born.

In many cases, decompilation is better. But there are some cases where \ 
decompilation has gotten worse. For lack of time (and interest) 3.0 bytecode \ 
suffered a hit. Possibly some code in the 3.x range did too. In time and with \ 
cleaner refactored code, this will come back.

Commit c90ff51 was a local maxiumum before, I started reworking the grammar to \ 
separate productions that were specific to loops versus those that are not in \ 
loops.
In the middle of that I added another grammar simplication to remove singleton \ 
productions of the form sstmts-> stmts. These were always was a bit ugly, and \ 
complicated output.

At any rate if decompilation fails, you can try c90ff51. Or another decompiler. \ 
unpyc37 is pretty good for 3.7. wibiti uncompyle2 is great for 2.7. pycdc is \ 
mediocre for Python before 3.5 or so, and not that good for the most recent \ 
Python. Generally these programs will give some sort of answer even if it isn't \ 
correct.

decompyle3 isn't that good for 3.7 and worse for 3.8, but right now it does \ 
things no other Python decompiler like unpyc37 or pycdc does. For example, \ 
decompyle3 handles variable annotations. As always, the issue trackers for the \ 
various programs will give you a sense for what needs to be done. For now, I've \ 
given up on reporting issues in the other decompilers because there are already \ 
enough issues reported, and they are just not getting fixed anyway.

Files:
RevisionActionfile
1.19modifypkgsrc/lang/py-uncompyle6/Makefile
1.12modifypkgsrc/lang/py-uncompyle6/PLIST
1.18modifypkgsrc/lang/py-uncompyle6/distinfo