General Guaranty Mortgage Co., Inc., Appellant, v. Laurie W. Tomlinson, District Director of Internal Revenue, Appellee, 335 F.2d 518 (5th Cir. 1964)

The Commissioner of Internal Revenue asserted a deficiency in income taxes against appellant mortgage company which, together with interest thereon, was duly paid. This appeal is from the decision of the District Court adverse to appellant on a suit for refund. A payment by First Federal Savings and Loan Association of St. Petersburg, Florida, to General Guaranty Insurance Company, the parent Corporation of appellant, in termination of two mortgage servicing contracts forms the crux of the controversy.

Appellant was engaged in the business of servicing mortgages owned by various financial institutions. About half of its portfolio consisted of First Federal mortgages. Its right to service the mortgages existed under two contracts outstanding since 1951 with First Federal.

General Guaranty Insurance Company acquired all of the outstanding capital stock of appellant mortgage company in January, 1956. Thereafter appellant serviced the mortgages in its portfolio as well as the mortgages of the insurance company. An audit by First Federal in the latter part of 1956 disclosed that certain terms of the 1951 servicing contracts were not being met by appellant. The chairman of First Federal had already determined that it could service its own mortgages for one half the amount being paid appellant. He also wanted the benefits to be gained from direct contact with the mortgagors. The servicing contracts contained no provision with respect to severance payment and the like in the event of termination. First Federal obtained the advice of counsel that the delinquencies on the part of appellant were grounds for termination, and it was then decided to terminate the contracts.

Upon notice of this fact being given, negotiations ensued between counsel, acting at the direction of the insurance company for its subsidiary, the appellant mortgage company, on the one side, and the chairman of and counsel for First Federal on the other. The sole issue had to do with reaching an agreement with respect to the terms of termination, and particularly as to what payment was to be made to appellant. These negotiations took place during December 1956 and the early part of January 1957. Seven-eighths of one percent of the outstanding balance due on the mortgages was finally agreed upon as the termination or cancellation fee. During the course of the negotiations, counsel representing appellant sought a delay in the termination of from ninety to one hundred twenty days for the stated purpose of gaining a tax advantage. However, on the trial he stated that this was by way of pretext only, and that his real purpose was to gain time within which to satisfy First Federal. To this end the servicing of the mortgages was to be transferred from the mortgage company to the insurance company in an effort to impress on First Federal that the mortgages were being serviced properly, and that it would be to its interest to permit the insurance company to continue the servicing.

Whatever the purpose of the delay, under the terms of the termination agreement, reached on January 16, 1957, termination was to be effective after March 1 but not later than May 1, 1957. With permission of First Federal, the mortgages and the records pertaining thereto were transferred by appellant to the insurance company on January 25, 1957. Thereafter the servicing was done by the insurance company, and it was paid the regular charge for that servicing by First Federal. Those sums are not in issue.

The termination agreement was consummated on May 1, 1957, and under its terms First Federal paid to the insurance company the sum of $162,480.83. The insurance company returned that payment on its tax return for the year 1957. The return reflected a net loss for the year of $169,853.10. The net operating loss carryover of the insurance company to the year 1958 was $432,272.25. Appellant mortgage company did not include any portion of the termination payment on its tax return, showing only a taxable income of $18,892.45, and federal income tax payable in the amount of $5,667.74. The basis for the deficiency assessment by the Commissioner was that the termination payment accrued for Federal income tax purposes to appellant mortgage company and not to the insurance company. The District Court agreed.

The record discloses an ample evidentiary base for the findings of fact entered by the court to the extent that they require no less than a conclusion that the substance of the termination transaction for income tax purposes was payment to appellant rather than to the parent insurance company. The termination of the contracts under the circumstances was a fait accompli when the decision to transfer the mortgages in question to the insurance company was made. All that remained was for the agreed payment due at the date of termination, to be selected by First Federal between March 1 and May 1, to be computed and paid. Counsel, acting both for the parent insurance company and appellant, represented to First Federal that the delay was sought for tax reasons. The chairman of First Federal agreed to the delay since some time would be necessarily required for First Federal to set up to service the mortgages.

The decision of the Supreme Court in Commissioner v. Court Holding Company, 1945, 324 U.S. 331, 65 S. Ct. 707, 89 L. Ed. 981, is controlling by analogy. See also Hindes v. United States, 5 Cir., 1964, 326 F.2d 150. In the Court Holding Company case the corporation had negotiated the sale of its sole asset, an apartment house. An oral agreement had been reached, and a down payment made. The corporation was then advised that the imposition of a large income tax would result from the sale. On the next day the corporation was liquidated with its stockholders receiving the apartment house in exchange for their stock. The sale was then made by the stockholders to the same vendees. The Supreme Court stated that the incidence of taxation depends upon the substance of a transaction, with the transaction being viewed as a whole, and that the true nature of a transaction is not to be disguised by mere formalisms existing solely to alter tax liabilities. The sale was treated as that of the corporation.

Here, it is true that evidence was offered to support a business purpose for the transfer of the mortgages from appellant to the insurance company, i. e., an effort to save the business, but the decision to terminate had already been made, and all terms therefor agreed upon. The transfer was thus, in effect, as in the Court Holding Company decision, after the fact, and cannot stand as a means to avoid the tax liability. Cf. Howell Turpentine Co. v. Commissioner, 5 Cir., 1947, 162 F.2d 319, distinguishing Court Holding Company on the basis that the negotiations, and subsequent sale were those of the stockholders who had previously determined to liquidate the corporation, and the sale was thus not to be attributed to the corporation for tax purposes.

With respect to the holding by the District Court that the sum realized from the termination was ordinary income rather than a capital gain, we have quite another story. After the decision of the District Court in this case, the decision in Bisbee-Baldwin Corp. v. Tomlinson, 5 Cir., 1963, 320 F.2d 929, was rendered on a very similar factual situation. There the mortgage servicing contracts which were terminated were regarded as consisting of a bundle of property rights. The right to receive servicing commissions was regarded as a substitute for future ordinary income and not subject to capital gains treatment. Other parts of the bundle were pointed to as being in the nature of capital assets. These included the contacts resulting from the mortgage correspondent relationship with the mortgagors. These contacts between the mortgage company and the mortgagors were said to act as a feeder for related businesses of the mortgage servicing company such as insurance, real estate and property management. Goodwill, described as embracing the tendency for future growth, as well as files and equipment fell into this category. The escrow accounts, as they enhanced the credit standing of the mortgage company were also referred to as a capital asset.

There we remanded for the District Court to allocate the amount of the termination price as between ordinary income and capital gains according to the realities of the transaction. See also United States v. Woolsey, 5 Cir., 1963, 326 F.2d 287 where the court spoke in terms of comminuting the payment into fragments so as to allocate it among the various assets sold. See Nelson Weaver Realty Co. v. Commissioner, 5 Cir., 1962, 307 F.2d 897, (dissenting opinion).

Whether this will be possible, or whether appellant is entitled to any relief by way of capital gains treatment will depend on the evidence. Further evidence, as was noted in Woolsey, may be required. It is enough to say that here the record does not preclude all capital gains treatment in view of the consideration accorded taxpayer in Bisbee-Baldwin Corp. v. Tomlinson, supra.

The judgment of the District Court to the extent that the payment by First Federal of the sum in termination of the mortgage servicing contracts was payment to the mortgage company and not to the insurance company for federal income tax purposes is affirmed. The case is remanded for further consideration in the light of what we have had to say with regard to possible allocation of the assets conveyed by way of termination as between ordinary income and capital assets. The existence or non-existence of capital assets, or of a conveyance of capital assets, and questions of valuation will be for the District Court.¹ 

Affirmed in part; reversed in part; remanded for further proceedings not inconsistent herewith.