Well I guess it's better now than later but you've definitely got a task ahead of you. I was once in a team of three responsible for a refactoring a product of similar size. It was procedural code but I'll describe some of the issues we had that will similarly apply.
We started at the bottom and started easing into it by picking functions that should have been highly reusable but weren't. We'd write a bunch of unit tests on the existing code (none existed at all!), but before long, we faced our first big problem--the existing code had bugs that had been laying dormant.
Do we fix them? If we do, then we've gone beyond a refactoring. So we'd log an issue with the existing code hoping to get a fixed and freshly tested code base, but of course management decided there were more important priorities than fixing bugs that had never surfaced. Understandable.
So we thought we'd try fixing the bugs in our new code. Then we discovered that these bugs in the original code made other code work, so really were 'conceptual bugs' rather than 'functional bugs'. Well maybe. There were occasional intermittent spasms in the original software that had never been tracked down.
So then we changed tack and decided to keep the bugs in place, as a true refactoring should do. It's easy to unintentionally introduce bugs, it's far harder to do it intentionally!
The next problem was that the code was in such as mess that the initial unit tests we wrote had to substantially change to cater for the refactoring. In other words, two moving targets. Not good. Just writing the tests was taking ages and lost us belief in the worthiness of the project. It really was something you just wanted to walk away from.
We found in the end we really had to tone down the extent of the refactoring if we were going to finish this millennium, which meant the codebase we dreamed of wouldn't be achieved. We declared that the most feasible solution was just to clean and trim the code and at least make it conceptually easier to understand for future developers to modify.
The reduced benefits of the limited refactoring was deemed not worth the effort by management, and given that similar reliability issues were being found in the hardware platform (embedded project), the company decided it was their chance to renew the entire product, with the software written from scratch, new language, objects. It was only the extensive system test specs in place from the original product that meant this had a chance.
构建一堆测试。不幸的是，这会花费大量时间，而且大多数管理者看不到任何价值；毕竟到目前为止，没有他们你也相处得很好。回到信仰问题是没有帮助的；在发生任何有用的事情之前，你仍然需要花费很多时间。如果他们确实让您构建测试，那么您将面临在重构时改进测试的问题；它们可能不会改变一点功能，但当您构建新的 API 时，测试将必须更改以匹配新的 API。这是重构代码库之外的额外工作。
Clearly the absence of tests is going to make people nervous when you attempt to refactor the code. Where will anybody get any faith that your refactoring doesn't break the application? Most of the answers you'll get, I think, will be "this is gonna be very hard and not very successful", and this is largely because you are facing a huge manual task and no faith in the answer.
There are only two ways out.
Build a bunch of tests. Unfortunately, this will cost a lot of time and most managers don't see any value; after all, you've gotten along without them so far. Pointing back to the faith question won't help; you're still using a lot of time before anything useful happens. If they do let you build tests, you'll have the problem of evolving the tests as you refactor; they may not change functionality one bit, but as you build new APIs the tests will have to change to match the new APIs. That's additional work beyond refactoring the code base.
Automate the refactoring process. If you apply trustworthy automated transformations, you can argue (often unsuccessfully) that the refactored code preserves the original system function. The way to beat the unsucessful argument is to write those tests (see first method) and apply the refactoring process to the application and the tests; as the application changes structures, the tests have to change too. But they are just application code from the point of view of automated machinery.
Not a lot of people do the latter; where do you get the tools that can do such things?
In fact, such tools exist. They are called program transformation tools and are used to carry out massive transformations on code. Think of these as tools for literally refactoring in the large; because of scale, they tend not to be interactive.
It does take effort to configure them for the task at hand; you have to write custom rules to accomplish your custom desired result. You likely can't do this in a week, but this is a lot less work than manually modifying a large system. And you should consider that you have 150 man-years invested in the existing software; it took that long to make the mess. It seems reasonable that "some" effort small in comparison should be OK.
I only know of 3 such tools that have a chance of working on real code: TXL, Stratego/XT, and our tool, the DMS Software Reengineering Toolkit. The first two are academic products (although TXL has been used for commercial activities in the past); DMS is commercial.
DMS has been used for a wide variety of large-scale software anaysis and massive transformation tasks. One task was automated translation between languages for the B-2 Stealth Bomber. Another, much closer to your refactoring problem, was automated architecting of a large-scale component-based system C++ for componentts, from a legacy proprietary RTOS with its idiosyncratic rules about how components are organized, to CORBA/RT in which the component APIs had to be changed from ad hoc structures to CORBA-style facet and receptacle interfaces as well as using CORBA/RT services in place of the legacy RTOS services. (These tasks were both done with 1-2 man-years of actual effort, by pretty smart and DMS-savvy guys).
There's still the test-construction problem (Both of these examples above had great system tests already).. Here I'm going to go out on a limb. I believe there is hope in getting such tools to automate test generation by instrumenting running code to collect function input-output results. We've built all kinds of instrumentation for source code (obviously you have to compile it after instrumentation) and think we know how to do this. YMMV.
There is something you do which is considerably less ambitious: identify the reusable parts of the code, by finding out what has been reused in the code. Most software systems contain a lot of cloned code (our experience is 10-20% [and I'm surprised by the PHP report of smaller numbers in another answer; I suspect they are using a weak clone detector). Cloned code is a hint of a missing abstraction in the application software. If you can find the clones and see how they vary, you can pretty easily see how to abstract them into functions (or whatever) to make them explicit and reusable.
Salion Inc. did clone detection and abstraction. The paper doesn't explore the abstraction activity; what Salion actually did was a periodic review of the detected clones, and manual remediation of the egregrious ones or those that made sense into (often library) methods. The net result was the code base actually shrank in size and the programmers became more effective because they had better ("more reusable") libraries.
They used our CloneDR, a tool for finding clones by using the program syntax as a guide. CloneDR finds exact clones and near misses (replacement of identifiers or statements) and provides a specific list of clone locatons and clone paramerizations, regardless of layout and comments. You can see clone reports for a number of languages at the link. (I'm the originator and author of CloneDR among my many hats).
Regarding the "small clone percentage" for the PHP project discussed in another answer: I don't know what was being used for a clone detector. The only clone detector focused on PHP that I know is PHPCPD, which IMHO is a terrible clone detector; it only finds exact clones if I understand the claimed implementation. See the PHP example at our site for comparative purposes.
This is exactly what we've been doing for web2project for the past couple years.. we forked from an existing system (dotproject) that had terrible metrics like high cyclomatic complexity (low: 17, avg: 27, high: 195M), lots of duplicate code (8% of overall code), and zero tests.
Since the split, we've reduced duplicate code (2.1% overall), reduced the total code (200kloc to 155kloc), added nearly 500 unit tests, and improved cyclomatic complexity (low: 1, avg: 11, high: 145M). Yes, we still have a ways to go.
发布评论
评论（3）
好吧，我想现在比以后更好，但你肯定有一项任务摆在你面前。我曾经在一个三人团队中负责重构类似规模的产品。这是程序代码，但我将描述我们遇到的一些同样适用的问题。
我们从底层开始，通过挑选那些本应高度可重用但实际上却没有的函数来逐渐进入它。我们对现有代码编写了一堆单元测试（根本不存在！），但不久之后，我们遇到了第一个大问题 - 现有代码存在一直处于休眠状态的错误。
我们修复它们吗？如果我们这样做了，那么我们就已经超越了重构。因此，我们会记录现有代码的问题，希望获得经过修复和新测试的代码库，但当然，管理层认为有比修复从未出现的错误更重要的优先事项。可以理解。
所以我们想尝试修复新代码中的错误。然后我们发现原始代码中的这些错误使其他代码正常工作，因此确实是“概念错误”而不是“功能错误”。好吧，也许吧。原始软件中偶尔会出现间歇性的痉挛，但从未被追踪到。
因此，我们改变了策略，决定将错误保留在适当的位置，就像真正的重构应该做的那样。无意中引入错误很容易，而有意引入错误则要困难得多！
下一个问题是代码非常混乱，我们编写的初始单元测试必须进行大幅更改才能适应重构。换句话说，两个移动的目标。不好。仅仅编写测试就花费了很长时间，并且让我们对这个项目的价值失去了信心。这确实是你只想摆脱的事情。
我们最终发现，如果我们要完成这个千年，我们确实必须降低重构的程度，这意味着我们梦想的代码库将无法实现。我们宣称，最可行的解决方案就是清理和修剪代码，至少使其在概念上更容易理解，以便未来的开发人员进行修改。
管理层认为有限重构所带来的收益降低不值得，并且考虑到在硬件平台（嵌入式项目）中发现了类似的可靠性问题，该公司决定这是他们更新整个产品的机会，包括软件从头开始编写，新的语言，对象。只有原始产品的广泛系统测试规范才意味着它有机会。
Well I guess it's better now than later but you've definitely got a task ahead of you. I was once in a team of three responsible for a refactoring a product of similar size. It was procedural code but I'll describe some of the issues we had that will similarly apply.
We started at the bottom and started easing into it by picking functions that should have been highly reusable but weren't. We'd write a bunch of unit tests on the existing code (none existed at all!), but before long, we faced our first big problem--the existing code had bugs that had been laying dormant.
Do we fix them? If we do, then we've gone beyond a refactoring. So we'd log an issue with the existing code hoping to get a fixed and freshly tested code base, but of course management decided there were more important priorities than fixing bugs that had never surfaced. Understandable.
So we thought we'd try fixing the bugs in our new code. Then we discovered that these bugs in the original code made other code work, so really were 'conceptual bugs' rather than 'functional bugs'. Well maybe. There were occasional intermittent spasms in the original software that had never been tracked down.
So then we changed tack and decided to keep the bugs in place, as a true refactoring should do. It's easy to unintentionally introduce bugs, it's far harder to do it intentionally!
The next problem was that the code was in such as mess that the initial unit tests we wrote had to substantially change to cater for the refactoring. In other words, two moving targets. Not good. Just writing the tests was taking ages and lost us belief in the worthiness of the project. It really was something you just wanted to walk away from.
We found in the end we really had to tone down the extent of the refactoring if we were going to finish this millennium, which meant the codebase we dreamed of wouldn't be achieved. We declared that the most feasible solution was just to clean and trim the code and at least make it conceptually easier to understand for future developers to modify.
The reduced benefits of the limited refactoring was deemed not worth the effort by management, and given that similar reliability issues were being found in the hardware platform (embedded project), the company decided it was their chance to renew the entire product, with the software written from scratch, new language, objects. It was only the extensive system test specs in place from the original product that meant this had a chance.
显然，当您尝试重构代码时，缺乏测试会让人们感到紧张。哪里会有人相信你的重构不会破坏应用程序？我认为，您得到的大多数答案将是“这将非常困难而且不会很成功”，这很大程度上是因为您面临着巨大的手动任务并且对答案没有信心。
只有两条出路。
构建一堆测试。不幸的是，这会花费大量时间，而且大多数管理者看不到任何价值；毕竟到目前为止，没有他们你也相处得很好。回到信仰问题是没有帮助的；在发生任何有用的事情之前，你仍然需要花费很多时间。如果他们确实让您构建测试，那么您将面临在重构时改进测试的问题；它们可能不会改变一点功能，但当您构建新的 API 时，测试将必须更改以匹配新的 API。这是重构代码库之外的额外工作。
自动化重构过程。如果您应用值得信赖的自动转换，您可以争辩（通常不成功）重构的代码保留了原始系统功能。击败不成功论点的方法是编写这些测试（请参阅第一种方法）并将重构过程应用于应用程序和测试；随着应用程序结构的改变，测试也必须改变。但从自动化机器的角度来看，它们只是应用程序代码。
做后者的人并不多。您从哪里获得可以完成此类操作的工具？
事实上，这样的工具是存在的。它们被称为程序转换工具，用于对代码进行大规模转换。
将它们视为大规模重构的工具；因为规模，
他们往往不具有互动性。
为手头的任务配置它们确实需要花费精力；您必须编写自定义规则来实现您的自定义所需结果。您可能无法在一周内完成此操作，但这比手动修改大型系统要少得多。您应该考虑到您在现有软件上投入了 150 人年；花了这么长时间才把事情弄得一团糟。相比之下，“一些”努力应该是可以的，这似乎是合理的。
我只知道 3 个这样的工具有机会在实际代码上工作：TXL、Stratego/XT，以及我们的工具 DMS 软件再造工具包。前两个是学术产品（尽管TXL过去曾用于商业活动）； DMS 是商业性的。
DMS 已广泛用于各种大型软件分析和大规模转换任务。其中一项任务是B-2 隐形轰炸机语言之间的自动翻译。另一个更接近重构问题的问题是针对组件的基于组件的大规模系统 C++ 的自动架构，从传统的专有 RTOS（具有关于组件如何组织的特殊规则）到 CORBA/RT（其中组件 API 具有从临时结构更改为 CORBA 风格的 Facet 和 Receptacle 接口，并使用 CORBA/RT 服务代替传统的 RTOS 服务。 （这些任务都是由非常聪明且精通 DMS 的人员花费 1-2 人年的实际努力完成的）。
仍然存在测试构建问题（上面的两个例子都已经进行了很好的系统测试）..在这里我要冒险一下。我相信通过检测运行代码来收集函数输入输出结果，有希望获得此类工具来自动生成测试。我们已经为源代码构建了各种类型的检测（显然您必须在检测后对其进行编译）并且认为我们知道如何做到这一点。 YMMV。
您所做的事情远没有那么雄心勃勃：通过找出代码中已重用的内容来识别代码的可重用部分。大多数软件系统都包含大量克隆代码（我们的经验是 10-20% [并且我对另一个答案中较小数字的 PHP 报告感到惊讶；我怀疑他们正在使用弱克隆检测器）。克隆代码暗示应用程序软件中缺少抽象。如果您可以找到克隆并查看它们如何变化，您可以很容易地了解如何将它们抽象为函数（或其他）以使它们明确且可重用。
Salion Inc.进行了克隆检测和提取。本文没有探讨抽象活动；而是探讨了抽象活动。 Salion 实际上所做的是定期审查检测到的克隆，并对异常克隆或对（通常是图书馆）方法有意义的克隆进行手动修复。最终结果是代码库的大小实际上缩小了，程序员变得更加高效，因为他们拥有更好的（“更可重用”）库。
他们使用了我们的 CloneDR，这是一种通过使用程序语法作为指导来查找克隆的工具。 CloneDR 查找精确的克隆和未遂事件（标识符或语句的替换），并提供克隆位置和克隆参数化的特定列表，无论布局和注释如何。您可以在链接中查看多种语言的克隆报告。 （我是CloneDR 的创始人和作者）。
关于另一个答案中讨论的 PHP 项目的“小克隆百分比”：我不知道克隆检测器使用了什么。据我所知，唯一专注于 PHP 的克隆检测器是 PHPCPD，恕我直言，这是一个糟糕的克隆检测器；如果我理解所声明的实现，它只会找到精确的克隆。出于比较目的，请参阅我们网站上的 PHP 示例。
Clearly the absence of tests is going to make people nervous when you attempt to refactor the code. Where will anybody get any faith that your refactoring doesn't break the application? Most of the answers you'll get, I think, will be "this is gonna be very hard and not very successful", and this is largely because you are facing a huge manual task and no faith in the answer.
There are only two ways out.
Build a bunch of tests. Unfortunately, this will cost a lot of time and most managers don't see any value; after all, you've gotten along without them so far. Pointing back to the faith question won't help; you're still using a lot of time before anything useful happens. If they do let you build tests, you'll have the problem of evolving the tests as you refactor; they may not change functionality one bit, but as you build new APIs the tests will have to change to match the new APIs. That's additional work beyond refactoring the code base.
Automate the refactoring process. If you apply trustworthy automated transformations, you can argue (often unsuccessfully) that the refactored code preserves the original system function. The way to beat the unsucessful argument is to write those tests (see first method) and apply the refactoring process to the application and the tests; as the application changes structures, the tests have to change too. But they are just application code from the point of view of automated machinery.
Not a lot of people do the latter; where do you get the tools that can do such things?
In fact, such tools exist. They are called program transformation tools and are used to carry out massive transformations on code.
Think of these as tools for literally refactoring in the large; because of scale,
they tend not to be interactive.
It does take effort to configure them for the task at hand; you have to write custom rules to accomplish your custom desired result. You likely can't do this in a week, but this is a lot less work than manually modifying a large system. And you should consider that you have 150 man-years invested in the existing software; it took that long to make the mess. It seems reasonable that "some" effort small in comparison should be OK.
I only know of 3 such tools that have a chance of working on real code: TXL, Stratego/XT, and our tool, the DMS Software Reengineering Toolkit. The first two are academic products (although TXL has been used for commercial activities in the past); DMS is commercial.
DMS has been used for a wide variety of large-scale software anaysis and massive transformation tasks. One task was automated translation between languages for the B-2 Stealth Bomber. Another, much closer to your refactoring problem, was automated architecting of a large-scale component-based system C++ for componentts, from a legacy proprietary RTOS with its idiosyncratic rules about how components are organized, to CORBA/RT in which the component APIs had to be changed from ad hoc structures to CORBA-style facet and receptacle interfaces as well as using CORBA/RT services in place of the legacy RTOS services. (These tasks were both done with 1-2 man-years of actual effort, by pretty smart and DMS-savvy guys).
There's still the test-construction problem (Both of these examples above had great system tests already).. Here I'm going to go out on a limb. I believe there is hope in getting such tools to automate test generation by instrumenting running code to collect function input-output results. We've built all kinds of instrumentation for source code (obviously you have to compile it after instrumentation) and think we know how to do this. YMMV.
There is something you do which is considerably less ambitious: identify the reusable parts of the code, by finding out what has been reused in the code. Most software systems contain a lot of cloned code (our experience is 10-20% [and I'm surprised by the PHP report of smaller numbers in another answer; I suspect they are using a weak clone detector). Cloned code is a hint of a missing abstraction in the application software. If you can find the clones and see how they vary, you can pretty easily see how to abstract them into functions (or whatever) to make them explicit and reusable.
Salion Inc. did clone detection and abstraction. The paper doesn't explore the abstraction activity; what Salion actually did was a periodic review of the detected clones, and manual remediation of the egregrious ones or those that made sense into (often library) methods. The net result was the code base actually shrank in size and the programmers became more effective because they had better ("more reusable") libraries.
They used our CloneDR, a tool for finding clones by using the program syntax as a guide. CloneDR finds exact clones and near misses (replacement of identifiers or statements) and provides a specific list of clone locatons and clone paramerizations, regardless of layout and comments. You can see clone reports for a number of languages at the link. (I'm the originator and author of CloneDR among my many hats).
Regarding the "small clone percentage" for the PHP project discussed in another answer: I don't know what was being used for a clone detector. The only clone detector focused on PHP that I know is PHPCPD, which IMHO is a terrible clone detector; it only finds exact clones if I understand the claimed implementation. See the PHP example at our site for comparative purposes.
这正是我们过去几年为 web2project 所做的事情。我们从现有的系统 (dotproject) 中分叉出来，该系统的指标很糟糕，例如高圈复杂度（低：17，平均：27，高：195M），很多重复代码（占全部代码的 8%）和零测试。
自拆分以来，我们减少了重复代码（总体为 2.1%），减少了总代码（200kloc 至 155kloc），添加了近 500 个单元测试，并提高了圈复杂度（低：1，平均：11，高：145M）。是的，我们还有很长的路要走。
我们的策略在我的幻灯片中有详细介绍：
http://caseysoftware.com/blog/phpbenelux-2011-recap - 项目分类与amp ;恢复;在这里：
http://www.phparch.com/2010/11/codeworks-2010- Slides/ - 单元测试策略；以及像这样的各种帖子：
http://caseysoftware.com/blog/technical-debt-doesn039t-disappear
只是警告你......一开始并不好玩。一旦您的指标开始改善，这可能会很有趣且令人满意，但这需要一段时间。
祝你好运。
This is exactly what we've been doing for web2project for the past couple years.. we forked from an existing system (dotproject) that had terrible metrics like high cyclomatic complexity (low: 17, avg: 27, high: 195M), lots of duplicate code (8% of overall code), and zero tests.
Since the split, we've reduced duplicate code (2.1% overall), reduced the total code (200kloc to 155kloc), added nearly 500 unit tests, and improved cyclomatic complexity (low: 1, avg: 11, high: 145M). Yes, we still have a ways to go.
Our strategy is detailed in my slides here:
http://caseysoftware.com/blog/phpbenelux-2011-recap - Project Triage & Recovery; and here:
http://www.phparch.com/2010/11/codeworks-2010-slides/ - Unit Testing Strategies; and in various posts like this one:
http://caseysoftware.com/blog/technical-debt-doesn039t-disappear
And just to warn you.. it's not fun at first. It can be fun and satisfying once your metrics start improving but that takes a while.
Good luck.