The Art of Computer Programming, Volume 4A: Combinatorial Algorithms, Part 1
Author: Donald E. Knuth
Date: 2014-09-12T00:00:00+00:00
Keywords: Computers & Technology; Programming; Programming Languages
Description: The Art of Computer Programming, Volume 4A: Combinatorial Algorithms, Part 1 Knuth’s multivolume analysis of algorithms is widely recognized as the definitive description of classical computer science. The first three volumes of this work have long comprised a unique and invaluable resource in programming theory and practice. Scientists have marveled at the beauty and elegance of Knuth’s analysis, while practicing programmers have successfully applied his “cookbook” solutions to their day-to-day problems. The level of these first three volumes has remained so high, and they have displayed so w
Publisher: Addison-Wesley Professional

About This eBook
ePUB is an open, industry-standard format for eBooks. However, support of ePUB and its many features varies across reading devices and applications. Use your device or app settings to customize the presentation to your liking. Settings that you can customize often include font, font size, single or double column, landscape or portrait mode, and figures that you can click or tap to enlarge. For additional information about the settings and features on your reading device or app, visit the device manufacturer’s Web site.

Many titles include programming code or configuration examples. To optimize the presentation of these elements, view the eBook in single-column, landscape mode and adjust the font size to the smallest setting. In addition to presenting code and configurations in the reflowable text format, we have included images of the code that mimic the presentation found in the print book; therefore, where the reflowable format may compromise the presentation of the code listing, you will see a “Click here to view code image” link. Click the link to view the print-fidelity code image. To return to the previous page viewed, click the Back button on your device or app.

The Art of Computer Programming

The Art of Computer Programming

Volume 4B / Combinatorial Algorithms, Part 2

Donald E. Knuth Stanford University

Boston • Columbus • New York • San Francisco • Amsterdam • Cape Town
Dubai • London • Madrid • Milan • Munich • Paris • Montréal • Toronto • Delhi • Mexico City
São Paulo • Sydney • Hong Kong • Seoul • Singapore • Taipei • Tokyo

Lyrics have been quoted on page 65 from the songs “Mississippi Mud,” written by Harry Barris and James Cavanaugh, and “Pick Yourself Up,” written by Dorothy Fields and Jerome Kern. Used by permission of Shapiro, Bernstein & Co, Inc.

The author and publisher have taken care in the preparation of this book, but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of the use of the information or programs contained herein.

For government sales inquiries, please contact governmentsales@pearsoned.com

For questions about sales outside the U.S., please contact intlcs@pearson.com

Visit us on the Web: informit.com/aw

Library of Congress Cataloging-in-Publication Data

Knuth, Donald Ervin, 1938-

The art of computer programming / Donald Ervin Knuth.

xviii, 714 p. 24 cm.

Includes bibliographical references and index.

Contents: v. 1. Fundamental algorithms. -- v. 2. Seminumerical
algorithms. -- v. 3. Sorting and searching. -- v. 4a. Combinatorial
algorithms, part 1. -- v. 4b. Combinatorial algorithms, part 2.

Contents: v. 4b. Combinatorial algorithms, part 2.

ISBN 978-0-201-89683-1 (v. 1, 3rd ed.)

ISBN 978-0-201-89684-8 (v. 2, 3rd ed.)

ISBN 978-0-201-89685-5 (v. 3, 2nd ed.)

ISBN 978-0-201-03804-0 (v. 4a)

ISBN 978-0-201-03806-4 (v. 4b)

1. Electronic digital computers--Programming.    2. Computer
algorithms. I. Title.

QA76.6.K64 1997

005.1--DC21                                                    97-2147

Internet page http://www-cs-faculty.stanford.edu/~knuth/taocp.xhtml contains current information about this book and related books.

See also http://www-cs-faculty.stanford.edu/~knuth/sgb.xhtml for information about The Stanford GraphBase, including downloadable software for dealing with the graphs used in many of the examples.

And see http://www-cs-faculty.stanford.edu/~knuth/mmix.xhtml for basic information about the MMIX computer.

Copyright © 2023 by Pearson Education, Inc.

All rights reserved. This publication is protected by copyright, and permission must be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system, or transmission in any form or by any means, electronic, mechanical, photocopying, recording, or likewise. For information regarding permissions, request forms, and the appropriate contacts with the Pearson Education Global Rights & Permissions Department, please visit www.pearson.com/permissions/.

ISBN-13 978-0-201-03806-4

ISBN-10        0-201-03806-4

First printing, October 2022

First digital release, October 2022

Preface

Begin at the beginning, and do not allow yourself to gratify
a mere idle curiosity by dipping into the book, here and there.

This would very likely lead to your throwing it aside,
with the remark “This is much too hard for me!,”

and thus losing the chance of adding a very large item
to your stock of mental delights.

— LEWIS CARROLL, in Symbolic Logic (1896)

Combinatorial algorithms are the methods that allow us to cope with problems that involve zillions of cases. The explosive growth in the knowledge of such techniques has meant that several volumes are needed to describe them. Thus my original plan to devote Volume 4 of The Art of Computer Programming to combinatorial algorithms has morphed into a plan to prepare Volumes 4A, 4B, and so on. This book is the second of that series, a sequel to Volume 4A.

In the preface to Volume 4A I explained why I was captivated by combinatorial algorithms soon after I fell in love with computers. “The art of writing such programs is especially important and appealing because a single good idea can save years or even centuries of computer time.”

Chapter 7 began in Volume 4A with a short review of graph theory and a longer discussion of “Zeros and Ones” (Section 7.1). That volume concluded with Section 7.2.1, “Generating Basic Combinatorial Patterns,” which was the first part of Section 7.2, “Generating All Possibilities.” Now the story continues, with the opening parts of Section 7.2.2, “Backtrack Programming.”

Backtracking is the name for an important body of techniques that have been a mainstay of combinatorial algorithms since the beginning. More than a third of this book is devoted to Section 7.2.2.1, which explores data structures whose links perform delightful dances. Such structures are ideally suited to backtrack programming in general, and to the “exact cover problem” (XC) in particular. The XC problem, also known as “set partitioning,” essentially asks for all ways to cover a set of items, by choosing appropriate subsets of items called options. Dozens of important applications turn out to be special cases of XC, and the method of choice for such problems is often to use dancing links.

While writing this material I learned to my surprise that an apparently innocuous extension of the classical XC problem leads to an enormous increase in the number of significant special cases. This extended problem, called XCC (for “exact covering with colors”), allows some of the items to receive various colors. Colored items are allowed to be covered by many different options, as long as the colors are compatible.

Spoiler alert: With dancing links, we can solve XCC problems almost as easily as XC problems! Therefore I believe that the study of XCC solvers, now in its infancy, is destined to become quite important, and I’ve done my best to introduce the subject here. There also are related methods for an even more general class of problems called MCC (“multiple covering with colors”), and for finding XCC solutions of minimum cost.

If you turn to a random page of Section 7.2.2.1, chances are good that you’ll find some sort of puzzle being discussed. The reason is that puzzles are by far the best means I know to illustrate the algorithms and techniques that are being introduced here. The point of a puzzle is easily grasped; and the fact that an extraordinary number of quite different puzzles all turn out to be special cases of XCC and MCC is significant in itself. Indeed, it becomes clear that the same ideas will solve many complex and harder-to-explain problems of the “real world.”

The new tools provided by dancing links allow me to emphasize the process of creating new puzzles, rather than simply to explain how to resolve puzzles that have already been posed. I’ve also tried my best to discuss the history of each puzzle type, and to give credit to the brilliant innovators who created them. As a result, I’m pleased that this book now contains, as a side-product of my attempts to teach computer methods, a treasure trove of information about recreational mathematics — from popular classics like edge-matching puzzles, or queen placement, or polyominoes, or the Soma cube, or rectangle dissections, or intriguing patterns of interlocking words, to more recent crazes like sudoku, slitherlink, masyu, and hitori.

I’ve had loads of fun writing other parts of these volumes, but without doubt Section 7.2.2.1 has been the funnest. And I know that my delight in good puzzles is shared by a significant number of leading computer scientists and mathematicians, who have told me that they chose their careers after having been inspired by such intellectual challenges.

Knuth likes to include in those books [The Art of Computer Programming]

as much recreational material as he can cram in.

— MARTIN GARDNER, Undiluted Hocus-Pocus (2013)

The second half of this book is devoted to Section 7.2.2.2, “Satisfiability,” which addresses one of the most fundamental problems in all of computer science: Given a Boolean function, can its variables be set to at least one pattern of 0s and 1s that will make the function true? This problem arises so often, people have given it a nickname, ‘SAT’.

Satisfiability might seem like an abstract exercise in understanding formal systems, but the truth is far different: Revolutionary methods for solving SAT problems emerged at the beginning of the twenty-first century, and they’ve led to game-changing applications in industry. These so-called “SAT solvers” can now routinely find solutions to practical problems that involve millions of variables and were thought until very recently to be hopelessly difficult.

Satisfiability is important chiefly because Boolean algebra is so versatile. Almost any problem can be formulated in terms of basic logical operations, and the formulation is particularly simple in a great many cases. Section 7.2.2.2 therefore begins with ten typical examples of widely different applications, and closes with detailed empirical results for a hundred different benchmarks. The great variety of these problems — all of which are special cases of SAT —is illustrated on pages 300 and 301 (which are my favorite pages in this book).

The story of satisfiability is the tale of a triumph of software engineering, blended with rich doses of beautiful mathematics. Section 7.2.2.2 explains how such a miracle occurred, by presenting complete details of seven SAT solvers, ranging from the small-footprint methods of Algorithms A and B to the industrial strength, state-of-the-art methods of Algorithms W, L, and C. (Well I have to hedge a little: New techniques are continually being discovered; hence SAT technology is ever-growing and the story is ongoing. But I do think that Algorithms W, L, and C compare reasonably well with the best algorithms of their class that were known in 2010. They’re no longer at the cutting edge, but they still are amazingly good.)

Wow — Sections 7.2.2.1 and 7.2.2.2 have turned out to be the longest sections, by far, in The Art of Computer Programming — especially Section 7.2.2.2. The SAT problem is evidently a killer app, because it is key to the solution of so many other problems. Consequently I can only hope that my lengthy treatment does not also kill off my faithful readers! As I wrote this material, one topic always seemed to flow naturally into another, so there was no neat way to break either section up into separate subsections. (And anyway the format of TAOCP doesn’t allow for a Section 7.2.2.1.3 or a Section 7.2.2.2.6.)

I’ve tried to ameliorate the reader’s navigation problem by adding sub-headings at the top of each right-hand page. Furthermore, as always, the exercises appear in an order that roughly parallels the order in which corresponding topics are taken up in the text. Numerous cross-references are provided between text, exercises, and illustrations, so that you have a fairly good chance of keeping in sync. I’ve also tried to make the index as comprehensive as possible.

Look, for example, at a “random” page — say page 264, which is part of the subsection about Monte Carlo algorithms. On that page you’ll see that exercises 302, 303, 299, and 306 are mentioned. So you can guess that the main exercises about Monte Carlo algorithms are numbered in the early 300s. (Indeed, exercise 306 deals with the important special case of “Las Vegas algorithms”; and the next exercises explore a fascinating concept called “reluctant doubling.”) This entire book is full of surprises and tie-ins to other aspects of computer science.

As in previous volumes, sections and subsections of the text are occasionally preceded by an asterisk (*), meaning that the topics discussed there are “advanced” and skippable on a first reading.

You might think that a 700-page book has probably been padded with peripheral material. But I constantly had to “cut, cut, cut” while writing it, because a great deal more is known! I found that new and potentially interesting-yet-unexplored topics kept popping up, more than enough to fill a lifetime; yet I knew that I must move on. So I hope that I’ve selected for treatment here a significant fraction of the concepts that will be the most important as time passes.

Every week I’ve been coming across fascinating new things
that simply cry out to be part of The Art.

— DONALD E. KNUTH (2008)

Most of this book is self-contained, although there are frequent tie-ins with the topics discussed in previous volumes. Low-level details of machine language programming have already been covered extensively; so the algorithms in the present book are usually specified only at an abstract level, independent of any machine. However, some aspects of combinatorial programming are heavily dependent on low-level details that didn’t arise before; in such cases, all examples in this book are based on the MMIX computer, which supersedes the MIX machine that was defined in early editions of Volume 1. Details about MMIX appear in a paperback supplement to that volume called The Art of Computer Programming, Volume 1, Fascicle 1, containing Sections 1.3.1´, 1.3.2´, etc.; they’re also available on the Internet, together with downloadable assemblers and simulators.

Another downloadable resource, a collection of programs and data called The Stanford GraphBase, is cited extensively in the examples of this book. Readers are encouraged to play with it, in order to learn about combinatorial algorithms in what I think will be the most efficient and most enjoyable way.

I wrote nearly a thousand computer programs while preparing this material, because I find that I don’t understand things unless I try to program them. Most of those programs were quite short, of course; but several of them are rather substantial, and possibly of interest to others. Therefore I’ve made a selection available by listing some of them on the following webpage:

http://www-cs-faculty.stanford.edu/~knuth/programs.xhtml

In particular you can download the programs DLX1, DLX2, DLX3, DLX5, DLX6, and DLX-PRE, which are the experimental versions of Algorithms X, C, M, C$, Z, and P, respectively, that were my constant companions while writing Section 7.2.2.1. Similarly, SAT0, SAT0W, SAT8, SAT9, SAT10, SAT11, SAT11K, SAT13 are the equivalents of Algorithms A, B, W, S, D, L, L′, C, respectively, in Section 7.2.2.2. Such programs will be useful for solving many of the exercises, if you don’t have access to other XCC solvers or SAT solvers. You can also download SATexamples.tgz from that page; it’s a collection of programs that generate data for all 100 of the benchmark examples discussed in the text, and many more.

Several exercises involve the lists of English words that I’ve used in preparing examples. You’ll need the data from
http://www-cs-faculty.stanford.edu/~knuth/wordlists.tgz
if you have the courage to work the exercises that use such lists.

Special Note: During the years that I’ve been preparing Volume 4, I’ve often run across basic techniques of probability theory that I would have put into Section 1.2 of Volume 1 if I’d been clairvoyant enough to anticipate them in the 1960s. Finally I realized that I ought to collect most of them together in one place, because the story of those developments is too interesting to be broken up into little pieces scattered here and there.

Therefore this book begins with a special tutorial and review of probability theory, in an unnumbered section entitled “Mathematical Preliminaries Redux.” References to its equations and exercises use the abbreviation ‘MPR’. (Think of the word “improvement.”)

Incidentally, just after the special MPR section, Section 7.2.2 begins intentionally on a left-hand page; and its illustrations are numbered beginning with Fig. 68. The reason is that Section 7.2.1 ended in Volume 4A on a right-hand page, and its final illustration was Fig. 67. My editor has decided to treat Chapter 7 as a single unit, even though it is being split into several physical volumes.

Special thanks are due to Nikolai Beluhov, Armin Biere, Niklas Eén, Marijn Heule, Holger Hoos, Wei-Hwa Huang, Svante Janson, Ernst Schulte-Geers, George Sicherman, Filip Stappers, and Udo Wermuth, for their detailed comments on my early attempts at exposition, as well as to dozens and dozens of other correspondents who have contributed crucial corrections. My editor at Addison–Wesley, Mark Taub, has expertly shepherded this series of books into the 21st century; and Julie Nahil, as senior content producer, has meticulously ensured that the highest publication standards have continued to be maintained. Thanks also to Tomas Rokicki for keeping my Dell workstation in shipshape order, as well as to Stanford’s InfoLab for providing extra computer power when that machine had reached its limits.

I happily offer a “finder’s fee” of $2.56 for each error in this book when it is first reported to me, whether that error be typographical, technical, or historical. The same reward holds for items that I forgot to put in the index. And valuable suggestions for improvements to the text are worth 32/c each. (Furthermore, if you find a better solution to an exercise, I’ll actually do my best to give you immortal glory, by publishing your name in subsequent printings:−)

Happy reading!

D. E. K.

Stanford, California

June 2022

A note on references

Several oft-cited journals and conference proceedings have special code names, which appear in the Index and Glossary at the close of this book. But the various kinds of IEEE Transactions are cited by including a letter code for the type of transactions, in boldface preceding the volume number. For example, ‘IEEE Trans. C-35’ means the IEEE Transactions on Computers, volume 35. The IEEE no longer uses these convenient letter codes, but the codes aren’t too hard to decipher: ‘EC’ once stood for “Electronic Computers,” ‘IT’ for “Information Theory,” ‘PAMI’ for “Pattern Analysis and Machine Intelligence,” and ‘SE’ for “Software Engineering,” etc.; ‘CAD’ meant “Computer-Aided Design of Integrated Circuits and Systems.”

A cross-reference such as ‘exercise 7.10–00’ points to a future exercise in Section 7.10 whose number is not yet known.

A note on notations

Simple and intuitive conventions for the algebraic representation of mathematical concepts have always been a boon to progress, especially when most of the world’s researchers share a common symbolic language. The current state of affairs in combinatorial mathematics is unfortunately a bit of a mess in this regard, because the same symbols are occasionally used with completely different meanings by different groups of people; some specialists who work in comparatively narrow subfields have unintentionally spawned conflicting symbolisms. Computer science — which interacts with large swaths of mathematics — needs to steer clear of this danger by adopting internally consistent notations whenever possible. Therefore I’ve often had to choose among a number of competing schemes, knowing that it will be impossible to please everyone. I have tried my best to come up with notations that I believe will be best for the future, often after many years of experimentation and discussion with colleagues, often flip-flopping between alternatives until finding something that works well. Usually it has been possible to find convenient conventions that other people have not already coopted in contradictory ways.

Appendix B is a comprehensive index to all of the principal notations that are used in the present book, inevitably including several that are not (yet?) standard. If you run across a formula that looks weird and/or incomprehensible, chances are fairly good that Appendix B will direct you to a page where my intentions are clarified. But I might as well list here a few instances that you might wish to watch for when you read this book for the first time:

- Hexadecimal constants are preceded by a number sign or hash mark. For example,#123 means (123)16.

- The “monus” operation x ∸ y, sometimes called dot-minus or saturating subtraction, yields max(0, x − y).

- The median of three numbers {x, y, z} is denoted by 〈xyz〉.

- The “two dots” notations (x..y), (x..y], [x..y), and [x..y] are used to denote intervals.

- A set such as {x}, which consists of a single element, is often denoted simply by x in contexts such as X ∪ x or X \ x.

- If n is a nonnegative integer, the number of 1-bits in n’s binary representation is νn. Furthermore, if n > 0, the leftmost and rightmost 1-bits of n are respectively 2λn and 2ρn. For example, ν 10 = 2, λ10 = 3, ρ10 = 1.

- The Cartesian product of graphs G and H is denoted by G □ H. For example, Cm □ Cn denotes an m × n torus, because Cn denotes a cycle of n vertices.

Notes on the Exercises

The exercises in this set of books have been designed for self-study as well as for classroom study. It is difficult, if not impossible, for anyone to learn a subject purely by reading about it, without applying the information to specific problems and thereby being encouraged to think about what has been read. Furthermore, we all learn best the things that we have discovered for ourselves. Therefore the exercises form a major part of this work; a definite attempt has been made to keep them as informative as possible and to select problems that are enjoyable as well as instructive.

In many books, easy exercises are found mixed randomly among extremely difficult ones. A motley mixture is, however, often unfortunate because readers like to know in advance how long a problem ought to take — otherwise they may just skip over all the problems. A classic example of such a situation is the book Dynamic Programming by Richard Bellman; this is an important, pioneering work in which a group of problems is collected together at the end of some chapters under the heading “Exercises and Research Problems,” with extremely trivial questions appearing in the midst of deep, unsolved problems. It is rumored that someone once asked Dr. Bellman how to tell the exercises apart from the research problems, and he replied, “If you can solve it, it is an exercise; otherwise it’s a research problem.”

Good arguments can be made for including both research problems and very easy exercises in a book of this kind; therefore, to save the reader from the possible dilemma of determining which are which, rating numbers have been provided to indicate the level of difficulty. These numbers have the following general significance:

Rating Interpretation

00 An extremely easy exercise that can be answered immediately if the material of the text has been understood; such an exercise can almost always be worked “in your head,” unless you’re multitasking.

10 A simple problem that makes you think over the material just read, but is by no means difficult. You should be able to do this in one minute at most; pencil and paper may be useful in obtaining the solution.

20 An average problem that tests basic understanding of the text material, but you may need about fifteen or twenty minutes to answer it completely. Maybe even twenty-five.

30 A problem of moderate difficulty and/or complexity; this one may involve more than two hours’ work to solve satisfactorily, or even more if the TV is on.

40 Quite a difficult or lengthy problem that would be suitable for a term project in classroom situations. A student should be able to solve the problem in a reasonable amount of time, but the solution is not trivial.

50 A research problem that has not yet been solved satisfactorily, as far as the author knew at the time of writing, although many people have tried. If you have found an answer to such a problem, you ought to write it up for publication; furthermore, the author of this book would appreciate hearing about the solution as soon as possible (provided that it is correct).

By interpolation in this “logarithmic” scale, the significance of other rating numbers becomes clear. For example, a rating of 17 would indicate an exercise that is a bit simpler than average. Problems with a rating of 50 that are subsequently solved by some reader may appear with a 40 rating in later editions of the book, and in the errata posted on the Internet (see page iv).

The remainder of the rating number divided by 5 indicates the amount of detailed work required. Thus, an exercise rated 24 may take longer to solve than an exercise that is rated 25, but the latter will require more creativity. All exercises with ratings of 46 or more are open problems for future research, rated according to the number of different attacks that they’ve resisted so far.

The author has tried earnestly to assign accurate rating numbers, but it is difficult for the person who makes up a problem to know just how formidable it will be for someone else to find a solution; and everyone has more aptitude for certain types of problems than for others. It is hoped that the rating numbers represent a good guess at the level of difficulty, but they should be taken as general guidelines, not as absolute indicators.

This book has been written for readers with varying degrees of mathematical training and sophistication; as a result, some of the exercises are intended only for the use of more mathematically inclined readers. The rating is preceded by an M if the exercise involves mathematical concepts or motivation to a greater extent than necessary for someone who is primarily interested only in programming the algorithms themselves. An exercise is marked with the letters “HM” if its solution necessarily involves a knowledge of calculus or other higher mathematics not developed in this book. An “HM” designation does not necessarily imply difficulty.

Some exercises are preceded by an arrowhead, “▶”; this designates problems that are especially instructive and especially recommended. Of course, no reader/student is expected to work all of the exercises, so those that seem to be the most valuable have been singled out. (This distinction is not meant to detract from the other exercises!) Each reader should at least make an attempt to solve all of the problems whose rating is 10 or less; and the arrows may help to indicate which of the problems with a higher rating should be given priority.

Several sections have more than 100 exercises. How can you find your way among so many? In general the sequence of exercises tends to follow the sequence of ideas in the main text. Adjacent exercises build on each other, as in the pioneering problem books of Pólya and Szegö. The final exercises of a section often involve the section as a whole, or introduce supplementary topics.

Solutions to most of the exercises appear in the answer section. Please use them wisely; do not turn to the answer until you have made a genuine effort to solve the problem by yourself, or unless you absolutely do not have time to work this particular problem. After getting your own solution or giving the problem a decent try, you may find the answer instructive and helpful. The solution given will often be quite short, and it will sketch the details under the assumption that you have earnestly tried to solve it by your own means first. Sometimes the solution gives less information than was asked; often it gives more. It is quite possible that you may have a better answer than the one published here, or you may have found an error in the published solution; in such a case, the author will be pleased to know the details. Later printings of this book will give the improved solutions together with the solver’s name where appropriate.

When working an exercise you may generally use the answers to previous exercises, unless specifically forbidden from doing so. The rating numbers have been assigned with this in mind; thus it is possible for exercise n + 1 to have a lower rating than exercise n, even though it includes the result of exercise n as a special case.

Summary of codes:

00 Immediate

10 Simple (one minute)

20 Medium (quarter hour)

▶        Recommended

30 Moderately hard

M      Mathematically oriented

40 Term project

HM  Requiring “higher math”

50 Research problem

Exercises

▶1. [00] What does the rating “M15 ” mean?

2. [10] Of what value can the exercises in a textbook be to the reader?

3. [HM45] Prove that every simply connected, closed 3-dimensional manifold is topologically equivalent to a 3-dimensional sphere.

The men that stood for office, noted for acknowledged worth,

And for manly deeds of honour, and for honourable birth;

Train’d in exercise and art, in sacred dances and in song,

All are ousted and supplanted by a base ignoble throng.

— ARISTOPHANES, The Frogs (405 B.C.)

Here mine aduice, shall be to those Artificers that will profite in this,
or any of my bookes nowe published, or that hereafter shall be,
firste confusely to reade them thorow; then with more iudgement,
and at the thirde readinge wittely to practise. So fewe thinges shall be vnknowen.

— LEONARDE DIGGES, A Boke named Tectonicon (1556)

Now I saw, tho’ too late, the Folly of
beginning a Work before we count the Cost,
and before we judge rightly of our own Strength to go through with it.

— DANIEL DEFOE, Robinson Crusoe (1719)

Contents

Preface

Notes on the Exercises

Mathematical Preliminaries Redux

Inequalities

Martingales

Tail inequalities from martingales

Applications

Statements that are almost sure, or even quite sure

Exercises

Chapter 7 — Combinatorial Searching

7.2. Generating All Possibilities

7.2.1. Generating Basic Combinatorial Patterns

7.2.2. Backtrack Programming

Data structures

Walker’s method

Permutations and Langford pairs

Word rectangles

Commafree codes

Dynamic ordering of choices

Sequential allocation redux

Lists for the commafree problem

A general mechanism for doing and undoing

Backtracking through commafree codes

Running time estimates

*Estimating the number of solutions

Factoring the problem

Historical notes

Exercises

7.2.2.1. Dancing links

Exact cover problems

Secondary items

Progress reports

Sudoku

Polyominoes

Polycubes

Factoring an exact cover problem

Color-controlled covering

Introducing multiplicity

*A new dance step

*Analysis of Algorithm X

*Analysis of matching problems

*Maintaining a decent focus

Exploiting local equivalence

*Preprocessing the options

Minimum-cost solutions

*Implementing the min-cost cutoffs

*Dancing with ZDDs

Summary

Historical notes

Exercises—First set

Exercises—Second set

Exercises—Third set

7.2.2.2. Satisfiability

Example applications

Backtracking algorithms

Random clauses

Resolution of clauses

Clause-learning algorithms

Monte Carlo algorithms

The Local Lemma

*Message-passing algorithms

*Preprocessing of clauses

Encoding constraints into clauses

Unit propagation and forcing

Symmetry breaking

Satisfiability-preserving maps

One hundred test cases

Tuning the parameters

Exploiting parallelism

History

Exercises

Answers to Exercises

Appendix A — Tables of Numerical Quantities

1. 1. Fundamental Constants (decimal)

2. 2. Fundamental Constants (hexadecimal)

3. 3. Harmonic Numbers, Bernoulli Numbers, Fibonacci Numbers

Appendix B — Index to Notations

Appendix C — Index to Algorithms and Theorems

Appendix D — Index to Combinatorial Problems

Appendix E — Answers to Puzzles in the Answers

Index and Glossary

We — or the Black Chamber — have a little agreement with [Knuth];

he doesn’t publish the real Volume 4 of The Art of Computer Programming,

and they don’t render him metabolically challenged.

— CHARLES STROSS, The Atrocity Archive (2001)

In books of this nature I can only suggest you keep it

as simple as the subject will allow.

— KODE VICIOUS (2012)

Mathematical Preliminaries Redux

Many parts of this book deal with discrete probabilities, namely with a finite or countably infinite set Ω of atomic events ω, each of which has a given probability Pr(ω), where

0≤Pr(ω)≤1and∑ω∈Ω⁢Pr(ω)=1.(1)

This set Ω, together with the function Pr, is called a “probability space.” For example, Ω might be the set of all ways to shuffle a pack of 52 playing cards, with Pr(ω)= 1/52! for every such arrangement.

An event is, intuitively, a proposition that can be either true or false with certain probability. It might, for instance, be the statement “the top card is an ace,” with probability 1/13. Formally, an event A is a subset of Ω, namely the set of all atomic events for which the corresponding proposition A is true; and

Pr(A) = ∑ω∈APr(ω)=∑ω∈ΩPr(ω)[ω∈A].(2)

A random variable is a function that assigns a value to every atomic event. We typically use uppercase letters for random variables, and lowercase letters for the values that they might assume; thus, we might say that the probability of the event X = x is Pr(X = x) = ∑ω∈Ω Pr(ω)[X(ω)= x]. In our playing card example, the top card T is a random variable, and we have Pr(T = 𝓠♠) = 1/52. (Sometimes, as here, the lowercase-letter convention is ignored.)

The random variables X_1, ... , Xk are said to be independent if

Pr(X1=x1⁢ and ⋅⋅⋅ and Xk=xk) = Pr (X1=x1) . . . Pr( Xk=xk)(3)

for all (x_1,...,xk). For example, if F and S denote the face value and suit of the top card T , clearly F and S are independent. Hence in particular we have Pr(T = 𝓠♠) = Pr(F = 𝓠) Pr(S = ♠). But T is not independent of the bottom card, B; indeed, we have Pr(T = t and B = b) ≠ 1/52^2 for any cards t and b.

A system of n random variables is called k-wise independent if no k of its variables are dependent. With pairwise (2-wise) independence, for example, we could have variable X independent of Y, variable Y independent of Z, and variable Z independent of X; yet all three variables needn’t be independent (see exercise 6). Similarly, k-wise independence does not imply (k + 1)-wise independence. But (k + 1)-wise independence does imply k-wise independence.

The conditional probability of an event A, given an event B, is

Pr(A⁢ |  B)=Pr(A∩B)Pr(B)=Pr(A and B)Pr(B),(4)

when Pr(B) > 0, otherwise it’s Pr(A). Imagine breaking the whole space Ω into two parts, Ω′ = B and Ω"=B¯=Ω⁢ \ B, with Pr(Ω′) = Pr(B) and Pr(Ω″) = 1 − Pr(B). If 0 < Pr(B) < 1, and if we assign new probabilities by the rules

we obtain new probability spaces Ω′ and Ω″, allowing us to contemplate a world where B is always true and another world where B is always false. It’s like taking two branches in a tree, each of which has its own logic. Conditional probability is important for the analysis of algorithms because algorithms often get into different states where different probabilities are relevant. Notice that we always have

Pr(A) = Pr(A⁢ | B) ⋅ Pr(B) + Pr(A⁢ | B¯)⋅Pr(B¯).(5)

The events A_1, ... , Ak are said to be independent if the random variables [A_1], ... , [Ak ] are independent. (Bracket notation applies in the usual way to events-as-statements, not just to events-as-subsets: [A] = 1if A is true, otherwise [A] = 0.) Exercise 20 proves that this happens if and only if

Pr(∩j∈JAj)=∏j∈JPr(Aj),      for all J⊆{1,.⁢.⁢.⁢,k}.(6)

In particular, events A and B are independent if and only if Pr(A|B) = Pr(A).

When the values of a random variable X are real numbers or complex numbers, we’ve defined its expected value E X in Section 1.2.10: We said that

E X=∑ω∈ΩX(ω) Pr(ω)=∑xx Pr(X=x),(7)

provided that this definition makes sense when the sums are taken over infinitely many nonzero values. (The sum should be absolutely convergent.) A simple but extremely important case arises when A is any event, and when X = [A] is a binary random variable representing the truth of that event; then

E[A] =∑ω∈Ω[A](ω) Pr(ω)=∑ω∈Ω[ω∈A]Pr(ω)=∑ω∈APr(ω)=Pr(A).(8)

We’ve also noted that the expectation of a sum, E(X_1 + ··· + Xk), always equals the sum of the expectations, (E X_1)+ ··· +(E Xk), whether or not the random variables Xj are independent. Furthermore the expectation of a product, E X_1 ...Xk, is the product of the expectations, (E X_1) ... (E Xk), if those variables do happen to be independent. In Section 3.3.2 we defined the covariance,
cover(X,Y)=E((X−E X) (Y−E Y))=(E XY)−(E X)(E Y),(9)

which tends to measure the way X and Y depend on each other. The variance, var(X), is covar(X, X); the middle formula in (9) shows why it is nonnegative whenever the random variable X takes on only real values.

All of these notions of expected value carry over to conditional expectation,

E(X| A)=∑ω∈AX(ω)Pr(ω)Pr(A)=∑x⁢ xPr(X=x⁢  and  A)Pr(A),(10)

conditioned on any event A, when we want to work in the probability space for which A is true. (If Pr(A) = 0, we define E(X |A) = E X.) One of the most important formulas, analogous to (5), is

E X=∑y⁢  E(X| Y=y)  Pr(Y=y)=∑y∑x⁢ x⁢ Pr(X=x⁢ |  Y=y) Pr(Y=y).(11)

Furthermore there’s also another important kind of conditional expectation: When X and Y are random variables, it’s often helpful to write ‘E(X | Y )’ for “the expectation of X given Y.” Using that notation, Eq. (11) becomes simply

E  X=E(E (X|Y)).(12)

This is a truly marvelous identity, great for hand-waving and for impressing outsiders — except that it can be confusing until you understand what it means.

In the first place, if Y is a Boolean variable, ‘E(X |Y )’ might look as if it means ‘E(X | Y = 1)’, thus asserting that Y is true, just as ‘E(X | A)’ asserts the truth of A in (10). No; that interpretation is wrong, quite wrong. Be warned.

In the second place, you might think of E(X | Y ) as a function of Y. Well, yes; but the best way to understand E(X | Y) is to regard it as a random variable. That’s why we’re allowed to compute its expected value in (12).

All random variables are functions of the atomic events ω. The value of E(X |Y ) at ω is the average of X(ω′) over all events ω′ such that Y (ω′)= Y (ω):

E(X|Y)(ω) = ∑ω′∈ΩX(ω′) Pr(ω′)[Y(ω′)]/Pr(Y=Y(ω)).(13)

Similarly, E(X |Y_1,...,Yr) averages over events with Yj(ω′)=Yj(ω) for 1≤j ≤r.

For example, suppose X_1 through Xn are binary random variables constrained by the condition that ν(X_1 ... Xn) = X_1 + ···+Xn = m, where m and n are constants with 0 ≤ m ≤ n; all (nm) such bit vectors X_1 ...Xn are assumed to be equally likely. Clearly E X_1 = m/n. But what is E(X_2 | X_1)? If X_1 = 0, the expectation of X_2 is m/(n − 1); otherwise that expectation is (m − 1)/(n − 1); consequently E(X_2 |X_1) = (m−X_1)/(n−1). And what is E(Xk |X_1,...,Xk−1)? The answer is easy, once you get used to the notation: If ν(X_1 ...Xk−1) = r, then Xk ...Xn is a random bit vector with ν(Xk ...Xn) = m − r; hence the average value of Xk will be (m − r)/(n + 1 − k) in that case. We conclude that

E Xk⁢ | X1,.⁢.⁢.⁢,Xk−1=m−v(X1.⁢.⁢.⁢ Xk−1)n+1−k,           for 1≤k≤n.(14)

The random variables on both sides of these equations are the same.

Inequalities

In practice we often want to prove that certain events are rare, in the sense that they occur with very small probability. Conversely, our goal is sometimes to show that an event is not rare. And we’re in luck, because mathematicians have devised several fairly easy ways to derive upper bounds or lower bounds on probabilities, even when the exact values are unknown.

We’ve already discussed the most important technique of this kind in Section 1.2.10. Stated in highly general terms, the basic idea can be formulated as follows: Let f be any nonnegative function such that f(x) ≥ s > 0 when x ∈ S. Then

Pr(X∈S)≤E f(X)/s,(15)

provided that Pr(X ∈ S) and E f(X) both exist. For example, f(x) = |x| yields

Pr(|X|≥m)≤ E |X|/m(16)

whenever m > 0. The proof is amazingly simple, because we obviously have

E f(X)≥Pr(X∈S)⋅s+Pr(X∉S)⋅0.(17)

Formula (15) is often called Markov’s inequality, because A. A. Markov discussed the special case f(x) = |x|a in Imp. Akad. Nauk (6) 1 (1907), 707–716. If we set f(x) = (x − E X)2, we get the famous 19th-century inequality of Bienaymé and Chebyshev:

Pr(|X−EX|≥r) ≤ var(X)/r2.(18)

The case f(x)= eax is also extremely useful.

Another fundamental estimate, known as Jensen’s inequality [Acta Mathematica 30 (1906), 175–193], applies to convex functions f; we’ve seen it so far only as a “hint” to exercise 6.2.2–36(!). The real-valued function f is said to be convex in an interval I of the real line, and −f is said to be concave in I, if
f(px+qy)≤pf(x)+qf(y)for all x, y∈I,(19)

whenever p ≥ 0, q ≥ 0, and p+q = 1. This condition turns out to be equivalent to saying that f″(x) ≥ 0 for all x ∈ I, if f has a second derivative f″. For example, the functions eax and x2n are convex for all constants a and all nonnegative integers n; and if we restrict consideration to positive values of x, then f(x) = xn is convex for all integers n (notably f(x) = 1/x when n = −1). The functions ln(1/x) and x ln x are also convex for x > 0. Jensen’s inequality states that
f(E X) ≤ E f(X)(20)

when f is convex in the interval I and the random variable X takes values only in I. (See exercise 42 for a proof.) For example, we have 1/ E X ≤ E(1/X) and ln E X ≥ Eln X and (E X)ln E X ≤ E(X ln X), when X is positive, since the function ln x is concave for x > 0. Notice that (20) actually reduces to the very definition of convexity, (19), in the special case when X = x with probability p and X = y with probability q = 1 − p.

Next on our list of remarkably useful inequalities are two classical results that apply to any random variable X whose values are nonnegative integers:

Pr(X>0) ≤E X;(“the first moment principle”)(21)

Pr(X>0) ≥(EX)2/(EX2).(“the second moment principle”)(22)

Formula (21) is obvious, because the left side is p_1 + p_2 + p_3 + ··· when pk is the probability that X = k, while the right side is p_1 +2p_2 +3p_3 + ···.

Formula (22) isn’t quite so obvious; it is p_1 + p_2 + p_3 + ··· on the left and (p_1 + 2p_2 + 3p_3 + ···)2/(p_1 + 4p_2 + 9p_3 + ···) on the right. However, as we saw with Markov’s inequality, there is a remarkably simple proof, once we happen to discover it: If X is nonnegative but not always zero, we have

E X2=E (X2| X>0) Pr(X>0) + E(X2|X=0) Pr(X=0)=E (X2| X>0) Pr(X>0)≥(E (X|X>0))2   Pr (X>0) = (EX)2/Pr(X>0).(23)

In fact this proof shows that the second moment principle is valid even when X is not restricted to integer values (see exercise 46). Furthermore the argument can be strengthened to show that (22) holds even when X can take arbitrary negative values, provided only that E X ≥ 0 (see exercise 47). See also exercise 118.

Exercise 54 applies (21) and (22) to the study of random graphs.

Another important inequality, which applies in the special case where X = X_1 + ··· + Xm is the sum of binary random variables Xj, was introduced more recently by S. M. Ross [Probability, Statistics, and Optimization (New York: Wiley, 1994), 185.190], who calls it the “conditional expectation inequality”:

Pr(X>0)≥∑j=1mE XjE(X|Xj =1).(24)

Ross showed that the right-hand side of this inequality is always at least as big as the bound (E X)2/(E X^2) that we get from the second moment principle (see exercise 50). Furthermore, (24) is often easier to compute, even though it may look more complicated at first glance.

For example, his method applies nicely to the problem of estimating a reliability polynomial, f(p_1,...,pn), when f is a monotone Boolean function; here pj represents the probability that component j of a system is “up.” We observed in Section 7.1.4 that reliability polynomials can be evaluated exactly, using BDD methods, when n is reasonably small; but approximations are necessary when f gets complicated. The simple example f(x_1,...,x_5) = x_1x_2x_3 ∨x_2x_3x_4 ∨ x_4x_5 illustrates Ross’s general method: Let (Y_1,...,Y_5) be independent binary random variables, with E Yj = pj; and let X = X_1+X_2+X_3, where X_1 = Y_1Y_2Y_3, X_2 = Y_2Y_3Y_4, and X_3 = Y_4Y_5 correspond to the prime implicants of f. Then Pr(X > 0) = Pr(f(Y_1,...,Y_5) = 1) = E f(Y_1,...,Y_5) = f(p_1,...,p_5), because the Y ’s are independent. And we can evaluate the bound in (24) easily:

Pr(X>0) ≥p1p2p31+p4+p4p5+p2p3p4p1+1+p5+p4p5p1p2p3+p2p3+1.(25)

If, for example, each pj is 0.9, this formula gives ≈ 0.848, while (E X)2/(E X^2) ≈ 0.847; the true value, p_1p_2p_3 + p_2p_3p_4 + p_4p_5 − p_1p_2p_3p_4 − p_2p_3p_4p_5, is 0.9558.

Many other important inequalities relating to expected values have been discovered, of which the most significant for our purposes in this book is the FKG inequality discussed in exercise 61. It yields easy proofs that certain events are correlated, as illustrated in exercise 62.

Martingales

A sequence of dependent random variables can be difficult to analyze, but if those variables obey invariant constraints we can often exploit their structure. In particular, the “martingale” property, named after a classic betting strategy (see exercise 67), proves to be amazingly useful when it applies. Joseph L. Doob featured martingales in his pioneering book Stochastic Processes (New York: Wiley, 1953), and developed their extensive theory.

The sequence 〈Zn〉 = Z_0, Z_1, Z_2, ... of real-valued random variables is called a martingale if it satisfies the condition

E (Zn+1|⁢ Z0⁢ ,.⁢.⁢.⁢,Zn)=Znfor all n≥0.(26)

(We also implicitly assume, as usual, that the expectations E Zn are well defined.) For example, when n = 0, the random variable E(Z_1 | Z_0) must be the same as the random variable Z_0 (see exercise 63).

Figure P illustrates George Pólya’s famous “urn model” [F. Eggenberger and G. Pólya, Zeitschrift für angewandte Math. und Mech. 3 (1923), 279–289], which is associated with a particularly interesting martingale. Imagine an urn that initially contains two balls, one red and one black. Repeatedly remove a randomly chosen ball from the urn, then replace it and contribute a new ball of the same color. The numbers (r, b) of red and black balls will follow a path in the diagram, with the respective local probabilities indicated on each branch.

One can show without difficulty that all n + 1 nodes on level n of Fig. P will be reached with the same probability, 1/(n + 1). Furthermore, the probability that a red ball is chosen when going from any level to the next is always 1/2. Thus the urn scheme might seem at first glance to be rather tame and uniform. But in fact the process turns out to be full of surprises, because any inequity between red and black tends to perpetuate itself. For example, if the first ball chosen is black, so that we go from (1, 1) to (1, 2), the probability is only 2 ln 2 − 1 ≈ .386 that the red balls will ever overtake the black ones in the future (see exercise 88).

One good way to analyze Pólya’s process is to use the fact that the ratios r/(r + b) form a martingale. Each visit to the urn changes this ratio either to (r +1)/(r +b+1) (with probability r/(r +b)) or to r/(r +b+1) (with probability b/(r+b)); so the expected new ratio is (rb+r^2 +r)/((r+b)(r+b+1)) = r/(r+b), no different from what it was before. More formally, let X_0 = 1, and for n > 0 let Xn be the random variable ‘[the nth ball chosen is red]’. Then there are X_0 + ··· + Xn red balls and X¯0+⋅⋅⋅+X¯n+1 black balls at level n of Fig. P; and the sequence 〈Zn〉 is a martingale if we define

Zn=(X0+⋅⋅⋅+Xn)/(n+2).(27)

In practice it’s usually most convenient to define martingales Z_0, Z_1, ... in terms of auxiliary random variables X_0, X_1, ... , as we’ve just done. The sequence 〈Zn〉 is said to be a martingale with respect to the sequence 〈Xn〉 if Zn is a function of (X_0,...,Xn) that satisfies

E (Zn+1| X0⁢ ,.⁢.⁢.⁢,Xn)=Znfor all n≥0.(28)

Fig. P. Pólya’s urn model. The probability of taking any downward path from (1, 1) to (r, b) is the product of the probabilities shown on the branches.

Furthermore we say that a sequence 〈Yn〉 is fair with respect to the sequence 〈Xn〉 if Yn is a function of (X_0,...,Xn) that satisfies the simpler condition

E (Yn+1| X0,.⁢.⁢.⁢ ,Xn)=0for all n≥0;(29)

and we call 〈Yn〉 fair whenever

E(Yn+1| Y0,.⁢.⁢.⁢ ,Yn)=0for all n≥0.(30)

Exercise 77 proves that (28) implies (26) and that (29) implies (30); thus an auxiliary sequence 〈Xn〉 is sufficient but not necessary for defining martingales and fair sequences.

Whenever 〈Zn〉 is a martingale, we obtain a fair sequence 〈Yn〉 by letting Y_0 = Z_0 and Yn = Zn − Zn−1 for n > 0, because the identity E(Yn+1 | Z_0,...,Zn) = E(Zn + 1 − Zn | Z_0,...,Zn) = Zn − Zn shows that 〈Yn〉 is fair with respect to 〈Zn〉. Conversely, whenever 〈Yn〉 is fair, we obtain a martingale 〈Zn〉 by letting Zn = Y_0 + ··· + Yn, because the identity E(Zn+1 | Y_0,...,Yn) = E(Zn + Yn+1 | Y_0,...,Yn) = Zn shows that 〈Zn〉 is a martingale with respect to 〈Yn〉. In other words, fairness and martingaleness are essentially equivalent. The Y ’s represent unbiased “tweaks” that change one Z to its successor.

It’s easy to construct fair sequences. For example, every sequence of independent random variables with mean 0 is fair. And if 〈Yn〉 is fair with respect to 〈Xn〉, so is the sequence 〈Yn′〉 defined by Yn′=fn(X0,.⁢.⁢.⁢,Xn−1)Yn when fn(X_0,...,Xn−1) is almost any function whatsoever! (We need only keep fn small enough that EYn′ is well defined.) In particular, we can let fn(X_0,...,Xn−1) = 0 for all large n, thereby making 〈Zn〉 eventually fixed.

A sequence of functions Nn(x_0,...,xn−1) is called a stopping rule if each value is either 0 or 1 and if Nn(x_0,...,xn−1) = 0 implies Nn+1(x_0,...,xn)= 0. We can assume that N_0 = 1. The number of steps before stopping, with respect to a sequence of random variables 〈Xn〉, is then the random variable

N=N1(X0)+N2(X0,X1)+N3(X0,X1,X2)+⋅⋅⋅.(31)

(Intuitively, Nn(x_0,...,xn−1) means [the values X_0 = x_0, ... , Xn−1 = xn−1 do not stop the process]; hence it’s really more about “going” than “stopping.”) Any martingale Zn = Y_0 + ··· + Yn with respect to 〈Xn〉 can be adapted to stop with this strategy if we change it to Zn′=Y0′+⋅⋅⋅+Yn′, where Yn′=Nn(X0,.⁢.⁢.⁢,Xn−1)Yn. Gamblers who wish to “quit when ahead” are using the stopping rule Nn+1(X0,...,Xn)=[Zn′≤0], when Zn′ is their current balance.

Notice that if the stopping rule always stops after at most m steps — in other words, if the function Nm(x_0,...,xm−1) is identically zero — then we have Zm′=ZN′, because Zn′ doesn’t change after the process has stopped. Therefore E ZN′=E Zm′=E Z0′=E Z0: No stopping rule can change the expected outcome of a martingale when the number of steps is bounded.

An amusing game of chance called Ace Now illustrates this optional stopping principle. Take a deck of cards, shuffle it and place the cards face down; then turn them face up one at a time as follows: Just before seeing the nth card, you are supposed to say either “Stop” or “Deal,” based on the cards you’ve already observed. (If n = 52 you must say “Stop.”) After you’ve decided to stop, you win $12 if the next card is an ace; otherwise you lose $1. What is the best strategy for playing this game? Should you hold back until you have a pretty good chance at the $12? What is the worst strategy? Exercise 82 has the answer.

Tail inequalities from martingales

The essence of martingales is equality of expectations. Yet martingales turn out to be important in the analysis of algorithms because we can use them to derive inequalities, namely to show that certain events occur with very small probability.

To begin our study, let’s introduce inequality into Eq. (26): A sequence 〈Zn〉 is called a submartingale if it satisfies

E(Zn+1| Z0,.⁢.⁢.⁢,Zn) ≥Znfor all   n≥0.(32)

Similarly, it’s called a supermartingale if ‘≥’ is changed to ‘≤’ in the left-hand part of this definition. (Thus a martingale is both sub- and super-.) In a submartingale we have E Z_0 ≤ E Z_1 ≤ E Z_2 ≤ ··· , by taking expectations in (32). A supermartingale, similarly, has ever smaller expectations as n grows. One way to remember the difference between submartingales and supermartingales is to observe that their names are the reverse of what you might expect.

Submartingales are significant largely because of the fact that they’re quite common. Indeed, if 〈Zn〉 is any martingale and if f is any convex function, then 〈f(Zn)〉 is a submartingale (see exercise 84). For example, the sequences 〈|Zn|〉 and 〈max(Zn,c)〈 and 〈Zn2〉 and 〈eZn〉 all are submartingales whenever 〈Zn〉 is known to be a martingale. If, furthermore, 〈Zn〉 is always positive, then 〈Zn3〉 and 〈1/Zn〉 and 〈ln(1/Zn)〉 and 〈Zn ln Zn〉, etc., are submartingales.

If we modify a submartingale by applying a stopping rule, it’s easy to see that we get another submartingale. Furthermore, if that stopping rule is guaranteed to quit within m steps, we’ll have E Zm≥E ZN=E ZN′=E Zm′. Therefore no stopping rule can increase the expected outcome of a submartingale, when the number of steps is bounded.

That comparatively simple observation has many important consequences. For example, exercise 86 uses it to give a simple proof of the so-called “maximal inequality”: If 〈Zn〉 is a nonnegative submartingale then

Pr(max(Z0,Z1,.⁢.⁢.⁢ ,Zn)≥x) ≤E Zn/x,for all   x>0.(33)

Special cases of this inequality are legion. For instance, martingales 〈Zn〉 satisfy

Pr(max(|Z0|,|Z1|,.⁢.⁢.⁢ ,|Zn|)≥x) ≤E |Zn|/x,for all   x>0;(34)

Pr(max(Z02,Z12,...,Zn2)≥x)≤E Zn2/x,         for all   x>0.

Relation (35) is known as Kolmogorov’s inequality, because A. N. Kolmogorov proved it when Zn = X_1 + ··· + Xn is the sum of independent random variables with E Xk = 0 and Xk=σk2 for 1 ≤ k ≤ n [Math. Annalen 99 (1928), 309– 311]. In that case var Zn=σ12+⋅⋅⋅+σn2=σ2, and the inequality can be written

Pr(|X1|<tσ, |X1+X2|<tσ,...,|X1+⋅⋅⋅+Xn|<tσ) ≥1−1/t2.(36)

Chebyshev’s inequality gives only Pr(|X_1 + ··· + Xn| < tσ) ≥ 1 − 1/t^2, which is a considerably weaker result.

Another important inequality applies in the common case where we have good bounds on the terms Y_1, ... , Yn that enter into the standard representation Zn = Y_0 + Y_1 + ··· + Yn of a martingale. This one is called the Hoeffding–Azuma inequality, after papers by W. Hoeffding [J. Amer. Statistical Association 58 (1963), 13–30] and K. Azuma [Tôhoku Math. Journal (2) 19 (1967), 357–367]. It reads as follows: If 〈Yn〉 is any fair sequence with an ≤ Yn ≤ bn when Y_0, Y_1, . . . , Yn−1 are given, then

Pr(Y1+⋅⋅⋅+Yn≥x) ≤e−2x2/((b1−a1)2+⋅⋅⋅+(bn−an)2).(37)

The same bound applies to Pr(Y_1 + ··· + Yn ≤ −x), since −bn ≤ −Yn ≤ −an; so

Pr(|Y1+⋅⋅⋅+Yn|≥x) ≤2e−2x2/((b1−a1)2+⋅⋅⋅+(bn−an)2).(38)

Exercise 90 breaks the proof of this result into small steps. In fact, the proof even shows that an and bn may be functions of {Y_0,...,Yn−1}.

Applications

The Hoeffding–Azuma inequality is useful in the analysis of many algorithms because it applies to “Doob martingales,” a very general class of martingales that J. L. Doob featured as Example 1 in his Stochastic Processes (1953), page 92. (In fact, he had already considered them many years earlier, in Trans. Amer. Math. Soc. 47 (1940), 486.) Doob martingales arise from any sequence of random variables 〈Xn〉, independent or not, and from any other random variable Q: We simply define

Zn=E(Q|X0,.⁢.⁢.⁢ ,Xn).(39)

Then, as Doob pointed out, the resulting sequence is a martingale (see exercise 91). In our applications, Q is an aspect of some algorithm that we wish to study, and the variables X_0, X_1, ... reflect the inputs to the algorithm. For example, in an algorithm that uses random bits, the X’s are those bits.

Consider a hashing algorithm in which t objects are placed into m random lists, where the nth object goes into list Xn; thus 1 ≤ Xn ≤ m for 1 ≤ n ≤ t, and we assume that each of the mt possibilities is equally likely. Let Q(x_1,...,xt) be the number of lists that remain empty after the objects have been placed into lists x_1, ... , xt, and let Zn = E(Q | X_1,...,Xn) be the associated Doob martingale. Then Z_0 = E Q is the average number of empty lists; and Zt = Q(X_1,...,Xt) is the actual number, in any particular run of the algorithm.

What fair sequence corresponds to this martingale? If 1 ≤ n ≤ t, the random variable Yn = Zn − Zn−1 is fn(X_1,...,Xn), where fn(x_1,...,xn) is the average of

Δ(x1,...⁢ ,xt)=∑x=1mPr(Xn=x)(Q(x1,.⁢ .⁢ .⁢ ,xn−1,xn,xn+1, .⁢ .⁢ .⁢ ,xt)                                                                              −⁢Q(x1,.⁢ .⁢ .⁢ ,xn−1,x,xn+1, .⁢ .⁢ .⁢ ,xt))(40)

taken over all mt−n values of (xn+1,...,xt).

In our application the function Q(x_1,...,xt) has the property that

|Q(x1,.⁢ .⁢ .⁢ ,xn−1,x′,xn+1,.⁢ .⁢ .⁢ ,xt)−Q(x1,.⁢ .⁢ .⁢ ,xn−1,x,xn+1,.⁢ .⁢ .⁢ ,xt)|≤1(41)

for all x and x′, because a change to any one hash address always changes the number of empty lists by either 1, 0, or −1. Consequently, for any fixed setting of the variables (x_1,...,xn−1,xn+1,...,xt), we have
maxxn⁢  Δ⁢ (x1, ...⁢ ,xt)⁢  ≤ maxxn  Δ⁢(x1, ...⁢ ,xt)+1.⁢  (42)

The Hoeffding–Azuma inequality (37) therefore allows us to conclude that

Pr(Zt−Z0⁢ ≥x) = Pr(Y1+⋅⋅⋅+Yt⁢ ≥x) ≤   e−2x2/t.(43)

Furthermore, Z_0 in this example is m(m − 1)t/mt, because exactly (m − 1)t of the mt possible hash sequences leave any particular list empty. And the random variable Zt is the actual number of empty lists when the algorithm is run. Hence we can, for example, set x=t⁢ ln f(t) in (43), thereby proving that

Pr(Zt≥(m−1)t/mt−1+t⁢  ln f(t)) ≤1/f(t)2.(44)

The same upper bound applies to Pr(Zt⁢ ≤(m−1)t/mt−1−t⁢  ln f(t)) .

Notice that the inequality (41) was crucial in this analysis. Therefore the strategy we’ve used to prove (43) is often called the “method of bounded differences.” In general, a function Q(x_1,...,xt) is said to satisfy a Lipschitz condition in coordinate n if we have

|Q(x1,.⁢ .⁢ .⁢ ,xn−1, x,xn+1,.⁢ .⁢ .⁢ ,xt )−Q(x1,.⁢ .⁢ .⁢ ,xn−1, x′,xn+1,.⁢ .⁢ .⁢ ,xt )|≤cn(45)

for all x and x′. (This terminology mimics a well-known but only slightly similar constraint that was introduced long ago into functional analysis by Rudolf Lipschitz [Crelle 63 (1864), 296–308].) Whenever condition (45) holds, for a function Q associated with a Doob martingale for independent random variables X_1, ... , Xt, we can prove that Pr(Y1+⋅⋅⋅+Yt⁢ ≥x)≤exp(−2x2/(c12+⋅⋅⋅+ct2)).

Let’s work out one more example, due to Colin McDiarmid [London Math. Soc. Lecture Notes 141 (1989), 148–188, §8(a)]: Again we consider independent integer-valued random variables X_1, ... , Xt with 1 ≤ Xn ≤ m for 1 ≤ n ≤ t; but this time we allow each Xn to have a different probability distribution. Furthermore we define Q(x_1,...,xt) to be the minimum number of bins into which objects of sizes x_1, ... , xt can be packed, where each bin has capacity m.

This bin-packing problem sounds a lot harder than the hashing problem that we just solved. Indeed, the task of evaluating Q(x_1,...,xt) is well known to be NP-complete [see M. R. Garey and D. S. Johnson, SICOMP 4 (1975), 397–411]. Yet Q obviously satisfies the condition (45) with cn = 1 for 1 ≤ n ≤ t. Therefore the method of bounded differences tells us that inequality (43) is true, in spite of the apparent difficulty of this problem!

The only difference between this bin-packing problem and the hashing problem is that we’re clueless about the value of Z_0. Nobody knows how to compute E Q(X_1,...,Xt), except for very special distributions of the random variables. However — and this is the magic of martingales — we do know that, whatever the value is, the actual numbers Zt will be tightly concentrated around that average.

If all the X’s have the same distribution, the values βt = E Q(X_1,...,Xt) satisfy βt+t′ ≤ βt+βt′, because we could always pack the t and t′ items separately. Therefore, by the subadditive law (see the answer to exercise 2.5–39), βt/t approaches a limit β as t → ∞. Still, however, random trials won’t give us decent bounds on that limit, because we have no good way to compute the Q function.

If only he could have enjoyed Martingale for its beauty and its peace
without being chained to it by this band of responsibility and guilt!

— P. D. JAMES, Cover Her Face (1962)

Statements that are almost sure, or even quite sure

Probabilities that depend on an integer n often have the property that they approach 0 or 1 as n → ∞, and special terminology simplifies the discussion of such phenomena. If, say, An is an event for which limn→∞ Pr(An) = 1, it’s convenient to express this fact in words by saying, “An occurs almost surely, when n is large.” (Indeed, we usually don’t bother to state that n is large, if we already understand that n is approaching infinity in the context of the current discussion.)

For example, if we toss a fair coin n times, we’ll find that the coin almost surely comes up heads more than .49n times, but fewer than .51n times.

Furthermore, we’ll occasionally want to express this concept tersely in formulas, by writing just ‘a.s.’ instead of spelling out the words “almost surely.” For instance, the statement just made about n coin tosses can be formulated as

.49n<X1+⋅⋅⋅+Xn⁢ <.51n⁢     a.s.,(46)

if X_1, ... , Xn are independent binary random variables, each with E Xj = 1/2. In general a statement such as “An a.s.” means that limn→∞ Pr(An) = 1; or, equivalently, that limn→∞⁢  Pr(A¯n)=0. It’s asymptotically almost sure.

If An and Bn are both a.s., then the combined event Cn = An ∩ Bn is also a.s., regardless of whether those events are independent. The reason is that Pr(C¯n) = Pr(A¯n⁢ ∪B¯n) ≤ Pr(A¯n⁢) + Pr(B¯n) , which approaches 0 as n → ∞.

Thus, to prove (46) we need only show that X_1 + ··· + Xn > .49n a.s. and that X_1 + ··· + Xn < .51n a.s., or in other words that Pr(X_1 + ··· + Xn ≤ .49n) and Pr(X_1 + ··· + Xn ≥ .51n) both approach 0. Those probabilities are actually equal, by symmetry between heads and tails; so we need only show that pn = Pr(X_1 + ··· + Xn ≤ .49n) approaches 0. And that’s no sweat, because we know from exercise 1.2.10–21 that pn ≤ e−.0001n.

In fact, we’ve proved more: We’ve shown that pn is superpolynomially small, namely that
pn=O(n−k)for all fixed numbers  K.(47)

When the probability of an event A¯n is superpolynomially small, we say that An holds “quite surely,” and abbreviate that by ‘q.s.’. In other words, we’ve proved

.49n<X1+⋅⋅⋅+Xn⁢ <.51n⁢   q.s.(48)

We’ve seen that the combination of any two a.s. events is a.s.; hence the combination of any finite number of a.s. events is also a.s. That’s nice, but q.s. events are even nicer: The combination of any polynomial number of q.s. events is also q.s. For example, if n^4 different people each toss n coins, it is quite sure that every one of them, without exception, will obtain between .49n and .51n heads!

(When making such asymptotic statements we ignore the inconvenient truth that our bound on the failure of the assertion, 2n^4e−.0001n in this case, becomes negligible only when n is greater than 700,000 or so.)

Exercises

1. [M21] (Nontransitive dice.) Suppose three biased dice with the respective faces

are rolled independently at random.

1. Show that Pr(A>B) = Pr(B>C) = Pr(C>A) = 5/9.

2. Find dice with Pr(A>B), Pr(B>C), Pr(C>A) all greater than 5/9.

3. If Fibonacci dice have Fm faces instead of just six, show that we could have

Pr(A>B) =Pr(B>C)=Fm−1/FmandPr(C>A) =Fm−1/Fm±1/Fm2.

2. [M32] Prove that the previous exercise is asymptotically optimum, in the sense that min(Pr(A>B), Pr(B>C), Pr(C>A)) < 1/φ, regardless of the number of faces.

3. [22] (Lake Wobegon dice.) Continuing the previous exercises, find three dice such that Pr(A>13(A+B+C))≥Pr(B>13(A+B+C))≥Pr(C>13(A+B+C))≥16/27. Each face of each die should be or or or or or .

4. [22] (Nontransitive Bingo.) Each player in the game of NanoBingo has a card containing four numbers from the set S = {1, 2, 3, 4, 5, 6}, arranged in two rows. An announcer calls out the elements of S, in random order; the first player whose card has a horizontal row with both numbers called shouts “Bingo!” and wins. (Or victory is shared when there are multiple Bingoes.) For example, consider the four cards

If the announcer calls “6, 2, 5, 1” when A plays against B, then A wins; but the sequence “1, 3, 2” would yield a tie. One can show that Pr(A beats B) = 336720, Pr(B beats A) = 312720, and Pr(A and B tie) = 72720. Determine the probabilities of all possible outcomes when there are (a) two (b) three (c) four different players using those cards.

▶5. [HM22] (T. M. Cover, 1989.) Common wisdom asserts that longer games favor the stronger player, because they provide more evidence of the relative skills.

However, consider an n-round game in which Alice scores A_1 + ‧ ‧ ‧ + An points while Bob scores B_1+ ‧ ‧ ‧ + Bn points. Here each of A_1, . . . , An are independent random variables with the same distribution, all representing Alice’s strength; similarly, each of B_1, . . . , Bn independently represent Bob’s strength (and are independent of the A’s). Suppose Alice wins with probability Pn.

1. Show that it’s possible to have P_1 = .99 but P_1000 < .0001.

2. Let mk = 2k^3, nk = 2k2+k, and qk = 2−k^2/D, where D = 2−0 + 2−1 + 2−4 + 2−9 + ‧ ‧ ‧ ≈ 1.56447. Suppose the random variable A takes the values (m_0, m_2, m_4, . . . ) with probabilities (q_0, q_2, q_4, . . . ); otherwise A = 0. Independently, the random variable B takes the values (m_1, m_3, m_5, . . .) with probabilities (q_1, q_3, q_5, . . . ); otherwise B = 0. What are Pr(A > B), Pr(A < B), and Pr(A = B)?

3. With the distributions in (b), prove that Pnk → [k even] as k→∞.

▶6. [M22] Consider random Boolean (or binary) vectors X_1 . . . Xn, where n ≥ 2, with the following distribution: The vector x_1 . . . xn occurs with probability 1/(n − 1)s^2 if x_1 + ‧ ‧ ‧ + xn = 2, with probability (n − 2)/(2n − 2) if x_1 + ‧ ‧ ‧ + xn = 0, and with probability 0 otherwise. Show that the components are pairwise independent (that is, Xi is independent of Xj when i ≠ j); but they are not k-wise independent for k > 2.

Also find a joint distribution, depending only on νx = x_1 +‧ ‧ ‧+ xn, that is k-wise independent for k = 2 and k = 3 but not k = 4.

7. [M30] (Ernst Schulte-Geers, 2012.) Generalizing exercise 6, construct a νx-based distribution that has k-wise but not (k + 1)-wise independence, given k ≥ 1.

▶8. [M20] Suppose the Boolean vector x_1 + ·· ·+ xn occurs with probability (2+(−1)νx)/2n+1, where νx = x_1 + ‧ ‧ ‧ + xn. For what k is this distribution k-wise independent?

9. [M20] Find a distribution of Boolean vectors x_1 . . . xn such that any two components are dependent; yet if we know the value of any xj, the remaining components are (n−1)-wise independent. Hint: The answer is so simple, you might feel hornswoggled.

▶10. [M21] Let Y_1, . . . , Ym be independent and uniformly distributed elements of {0, 1, . . . , p − 1}, where p is prime. Also let Xj = (jm + Y_1jm−1 + ‧ ‧ ‧ + Ym) mod p, for 1 ≤ j ≤ n. For what k are the X’s k-wise independent?

11. [M20] If X_1, . . . , X2n are independent random variables with the same discrete distribution, and if α is any real number whatsoever, prove that

Pr(|X1+⋅⋅⋅+X2n2n−α|≤X1+⋅⋅⋅+Xnn−α|)>12.

12. [21] Which of the following four statements are equivalent to the statement that Pr(A|B) > Pr(A)? (i) Pr(B |A) > Pr(B); (ii) Pr(A|B) > Pr(A|B¯); (iii) Pr(B|A) > Pr(B|A¯); (iv) Pr(A¯|B¯) > Pr(A¯|B).

13. [15] True or false: Pr(A|C) > Pr(A) if Pr(A|B) > Pr(A) and Pr(B|C) > Pr(B).

14. [10] (Thomas Bayes, 1763.) Prove the “chain rule” for conditional probability:

Pr(A_1 ∩ ··· ∩ An) = Pr(A_1)Pr(A_2|A_1)... Pr(An| A_1 ∩···∩ An−1).

15. [12] True or false: Pr(A|B ∩ C)Pr(B|C) = Pr(A ∩ B|C).

16. [M15] Under what circumstances is Pr(A | B) = Pr(A ∪ C| B)?

▶17. [15] Evaluate the conditional probability Pr(T is an ace |B = Q ♠) in the playing card example of the text, where T and B denote the top and bottom cards.

18. [20] Let M and m be the maximum and minimum values of the random variable X. Prove that var X ≤ (M − E X)(E X − m).

▶19. [HM28] Let X be a random nonnegative integer, with Pr(X = x) = 1/2x + 1, and suppose that X = (...X_2X_1X_0)2 and X + 1 = ( ...Y_2Y_1Y_0)2 in binary notation.

1. What is E Xn? Hint: Express this number in the binary number system.

2. Prove that the random variables {X_0,X_1,...,Xn−1} are independent.

3. Find the mean and variance of S = X_0 + X_1 + X_2 + ···.

4. Find the mean and variance of R = X_0 ⊕ X_1 ⊕ X_2 ⊕ ···.

5. Let π = (11.p_0p_1p_2 ...)2. What is the probability that Xn = pn for all n ≥ 0?

6. What is E Yn? Show that Y_0 and Y_1 are not independent.

7. Find the mean and variance of T = Y_0 + Y_1 + Y_2 + ···.

20. [M18] Let X_1, ..., Xk be binary random variables for which we know that E(∏j∈J Xj) = ∏j∈J E Xj for all J ⊆ {1,...,k}. Prove that the X’s are independent.

21. [M20] Find a small-as-possible example of random variables X and Y that satisfy covar(X, Y) = 0, that is, E XY = (E X)(E Y), although they aren’t independent.

▶22. [M20] Use Eq. (8) to prove the “union inequality”

Pr(A_1 ∪···∪ An) ≤ Pr(A_1) + ··· + Pr(An).

▶ 23. [M21] If each Xk is an independent binary random variable with E Xk = p, the cumulative binomial distribution Bm,n(p) is the probability that X_1 + ··· + Xn ≤ m. Thus it’s easy to see that Bm,n(p)=∑k=0m(nk)pk(1−p)n−k.

Show that Bm,n(p) is also equal to ∑k=0m(n−m−1+kk)pk(1−p)n−m, for 0 ≤ m ≤ n. Hint: Consider the random variables J_1, J_2, ... , and T defined by the rule that Xj = 0 if and only if j has one of the T values {J_1, J_2, ..., JT}, where 1 ≤ J_1 < J_2 < ··· < JT ≤ n. What is Pr(T ≥ r and Jr = s)?

▶24. [HM28] The cumulative binomial distribution also has many other properties.

1. Prove that Bm,n(p)=(n−m)(nm)∫p1  xm(1−x)n−1−mdx, for 0 ≤ m < n.

2. Use that formula to prove that Bm,n⁢(m/n)>12, for 0 ≤ m < n/2. Hint: Show that ∫0m/n xm(1−x)n−1−mdx<∫m/n1 xm(1−x)n−1−mdx.

3. Show furthermore that Bm,n⁢(m/n)>12 when n/2 ≤ m ≤ n. [Thus m is the median value of X_1 + ··· + Xn, when p = m/n and m is an integer.]

25. [M25] Suppose X_1, X_2, ... are independent random binary variables, with means E Xk = pk. Let ((nk)) be the probability that X_1 + ··· + Xn = k; thus ((nk))=pn((n−1k−1))+qn((n−1k))=[zk](q1+p1z)⋅⋅⋅(qn+pn⁢ z) .⁢⁢ .⁢⁢.(qn+pn⁢ z)⁢, where qk = 1 − pk.

1. Prove that ((nk))≥((nk+1)), if pj ≤ (k +1)/(n + 1) for 1 ≤ j ≤ n.

2. Furthermore ((nk))≤(nk)pkqn−k, if pj ≤ p ≤ k/n for 1 ≤ j ≤ n.

26. [M27] Continuing exercise 25, prove that ((nk))2≥((nk−1))((nk+1))(1+1k)(1+1n−k) for 0 < k < n. Hint: Consider rn,k=((nk))/(nk).

27. [M22] Find an expression for the generalized cumulative binomial distribution ∑k=0m((nk)) that is analogous to the alternative formula in exercise 23.

28. [HM28] (W. Hoeffding, 1956.) Let X = X_1 + ··· + Xn and p_1 + ··· + pn = np in exercise 25, and suppose that E g(X)≤∑k=0n g(k)((nk)) for some function g.

1. Prove that E g(X)≤∑k=0n g(k)(nk)pk(1−p)n−k if g is convex in [0 ..n].

2. If g isn’t convex, show that the maximum of E g(X), over all choices of {p_1,...,pn} with p_1 + ··· + pn = np can always be attained by a set of probabilities for which at most three distinct values {0,a, 1} occur among the pj.

3. Furthermore ∑k=0m ((nk))≤Bm,n⁢ (p), whenever p_1 + ··· + pn = np ≥ m + 1.

29. [HM29] (S. M. Samuels, 1965.) Continuing exercise 28, prove that we have Bm,n(p) ≥ ((1 − p)(m +1)/((1 − p)m + 1))n − m whenever np ≤ m + 1.

30. [HM34] Let X_1, ... , Xn be independent random variables whose values are nonnegative integers, where EXk = 1 for all k, and let p = Pr(X_1 + ··· + Xn ≤ n).

1. What is p, if each Xk takes only the values 0 and n + 1?

2. Show that, in any set of distributions that minimize p, each Xk assumes only two integer values, 0 and mk, where 1 ≤ mk ≤ n + 1.

3. Furthermore we have p > 1/e, if each Xk has the same two-valued distribution.

▶31. [M20] Assume that A_1, ... , An are random events such that, for every subset I ⊆ {1, ...,n}, the probability Pr(∩i∈I Ai) that all Ai for i ∈ I occur simultaneously is πI; here πI is a number with 0 ≤ πI ≤ 1, and π∅ = 1. Show that the probability of any combination of the events, Pr(f([A_1],..., [An])) for any Boolean function f, can be found by expanding f’s multilinear reliability polynomial f([A_1],..., [An]) and replacing each term ∏i∈I [Ai] by πI. For example, the reliability polynomial of x_1 ⊕x_2 ⊕x_3 is x_1 + x_2 + x_3 − 2x_1x_2 − 2x_1x_3 − 2x_2x_3 + 4x_1x_2x_3; hence Pr([A_1] ⊕ [A_2] ⊕ [A_3]) = π_1 + π_2 + π_3 − 2π_12 − 2π_13 − 2π_23 +4π_123. (Here ‘π_12’ is short for π{1,2}, etc.)

32. [M21] Not all sets of numbers πI in the preceding exercise can arise in an actual probability distribution. For example, if I ⊆ J we must have πI ≥ πJ . What is a necessary and sufficient condition for the 2n values of πI to be legitimate?

33. [M20] Suppose X and Y are binary random variables whose joint distribution is defined by the probability generating function G(w, z) = E(wXzY) = pw + qz + rwz, where p, q, r > 0 and p + q + r = 1. Use the definitions in the text to compute the probability generating function E(zE(X|Y)) for the conditional expectation E(X | Y ).

34. [M17] Write out an algebraic proof of (12), using the definitions (7) and (13).

▶35. [M22] True or false: (a) E(E(X | Y) | Y ) = E(X | Y ); (b) E(E(X | Y) | Z) = E(X | Z).

36. [M21] Simplify the formulas (a) E(f(X) | X); (b) E (f(Y )E(g(X) | Y)).

37. [M20] Suppose X_1 ...Xn is a random permutation of {1,...,n}, with every permutation occurring with probability 1/n!. What is E(Xk | X_1,...,Xk−1)?

38. [M26] Let X_1 ...Xn be a random restricted growth string of length n, each with probability 1/ϖn (see Section 7.2.1.5). What is E(Xk | X_1,...,Xk−1)?

▶39. [HM21] A hen lays N eggs, where Pr(N = n) = e−μμn/n! obeys the Poisson distribution. Each egg hatches with probability p, independent of all other eggs. Let K be the resulting number of chicks. Express (a) E(K | N), (b) E K, and (c) E(N | K) in terms of N, K, μ, and p.

40. [M16] Suppose X is a random variable with X ≤ M, and let m be any value with m < M. Show that Pr(X > m) ≥ (E X − m)/(M − m).

41. [HM21] Which of the following functions are convex in the set of all real numbers x? (a) |x|a, where a is a constant; (b) ∑k≥nxk/k!, where n ≥ 0 is an integer; (c) ee|x|; (d) f(x)[x ∈ I]+ ∞ [x /∈ I ], where f is convex in the interval I.

42. [HM21] Prove Jensen’s inequality (20).

▶43. [M18] Use (12) and (20) to strengthen (20): If f is convex in I and if the random variable X takes values in I, then f(E X) ≤ E (f(E(X | Y ))) ≤ E f(X).

▶44. [M25] If f is convex on the real line and if E X = 0, prove that E f(aX) ≤ E f(bX) whenever 0 ≤ a ≤ b.

45. [M18] Derive the first moment principle (21) from Markov’s inequality (15).

46. [M15] Explain why E(X^2 | X > 0) ≥ (E(X | X > 0))2 in (23).

47. [M15] If X is random and Y = max(0, X), show that E Y ≥ E X and E Y^2 ≤ E X^2.

▶48. [M20] Suppose X_1, ... , Xn are independent random variables with E Xk = 0 and E Xk2=σk2 for 1 ≤ k ≤ n. Chebyshev’s inequality tells us that Pr(|X_1+···+Xn| ≥ a) ≤ (σ12+⋅⋅⋅+σn2)/a2; show that the second moment principle gives a somewhat better one-sided estimate, Pr(X1+⋅⋅⋅+Xn≥a)  ≤(σ12+⋅⋅⋅+σn2)/(a2+σ12+⋅⋅⋅+σn2), if a ≥ 0.

49. [M20] If X is random and ≥ 0, prove that Pr(X = 0) ≤ (E X^2)/(E X)2 − 1.

▶50. [M27] Let X = X_1 + ··· + Xm be the sum of binary random variables, with E Xj = pj. Let J be independent of the X’s, and uniformly distributed in {1,...,m}.

1. Prove that Pr(X>0) = ∑j=1mE(Xj/X|Xj>0)⋅Pr(Xj>0).

2. Therefore (24) holds. Hint: Use Jensen’s inequality with f(x) = 1/x.

3. What are Pr(XJ = 1) and Pr(J = j | XJ = 1)?

4. Let tj = E(X | J = j and XJ = 1). Prove that E X2=∑j=1mpjtj.

5. Jensen’s inequality now implies that the right side of (24) is ≥ (E X)2/(E X^2).

▶51. [M21] Show how to use the conditional expectation inequality (24) to obtain also an upper bound on the value of a reliability polynomial, and apply your method to the case illustrated in (25).

52. [M21] What lower bound does inequality (24) give for the reliability polynomial of the symmetric function S≥k(x_1,...,xn), when p_1 = ··· = pn = p?

53. [M20] Use (24) to obtain a lower bound for the reliability polynomial of the non-monotonic Boolean function f(x1,.⁢.⁢.⁢,x6)=x1x2x¯3⁢  ∨x2x3x¯4⁢ ∨⋅⋅⋅∨ x5x6x¯1⁢ ∨x6x1x¯2.

54. [M22] Suppose each edge of a random graph on the vertices {1,...,n} is present with probability p, independent of every other edge. If u, v, w are distinct vertices, let Xuvw be the binary random variable [{u, v, w} is a 3-clique]; thus Xuvw = [u―v][u―w][v―w], and E Xuvw = p^3. Also let X = ∑1≤u<v<w≤n Xuvw be the total number of 3-cliques. Use the (a) first and (b) second moment principle to derive bounds on the probability that the graph contains at least one 3-clique.

55. [23] Evaluate the upper and lower bounds in the previous exercise numerically in the case n = 10, and compare them to the true probability, when (a) p = 1/2; (b) p = 1/10.

56. [HM20] Evaluate the upper and lower bounds of exercise 54 asymptotically when p = λ/n and n → ∞.

▶57. [M21] Obtain a lower bound for the probability in exercise 54(b) by using the conditional expectation inequality (24) instead of the second moment principle (22).

58. [M22] Generalizing exercise 54, find bounds on the probability that a random graph on n vertices has a k-clique, when each edge has probability p.

▶59. [HM30] (The four functions theorem.) The purpose of this exercise is to prove an inequality that applies to four sequences 〈an〉, 〈bn〉, 〈cn〉, 〈dn〉 of nonnegative numbers:

aj⁢ bk ≤cj|k⁢ dj&k⁢          for 0 ≤j,k<∞⁢          implies   ∑j=0∞⁢  ∑ k=0∞ajbk≤∑j=0∞⁢  ∑ k=0∞cjdk.(*)

(The sums will be ∞ if they don’t converge.) Although the inequality might appear at first to be merely a curiosity, of interest only to a few lovers of esoteric formulas, we shall see that it’s a fundamental result with many applications of great importance.

1. Prove the special case where aj = bj = cj = dj = 0 for j ≥ 2, namely that
a0b0≤c0d0,a0b1≤c1d0,        a1b0≤c1d0,         and        a1b1≤c1d1                        implies     (a0+a1)  (b0+b1) ≤(c0+c1) (d0+d1).

Can equality hold in the first four relations but not in the last one? Can equality hold in the last relation but not in the first four?

2. Use that result to prove (*) when aj = bj = cj = dj = 0 for all j ≥ 2n, given n > 0.

3. Conclude that (*) is true in general.

▶60. [M21] If 𝓕 is a family of sets, and if α is a function that maps sets into real numbers, let α(𝓕) = ∑S∈𝓕 α(S). Suppose 𝓕 and 𝓖 are finite families of sets for which nonnegative set functions α, β, γ, and δ have been defined with the property that
α(S) β(T) ≤ γ(S ∪ T) δ(S ∩ T)     for all S ∈ 𝓕 and T ∈ 𝓖.

1. Use exercise 59 to prove that α(𝓕)β(𝓖) ≤ γ(𝓕 𝓖) δ(𝓕 𝓖).

2. In particular, |𝓕| |𝓖| ≤ |𝓕 ⊔ 𝓖| |𝓕 ⊓ 𝓖| for all families 𝓕 and 𝓖.

▶61. [M28] Consider random sets in which S occurs with probability μ(S), where
μ(S)≥0⁢         and    μ(S)μ(T)≤ μ(S∪T) μ(S∩T)             for all sets S⁢   and S  and T. (**)

Assume also that U = ∪μ(S)>0 S is a finite set.

Prove the FKG inequality (which is named for C. M. Fortuin, P. W. Kasteleyn, and J. Ginibre): If f and g are real-valued set functions, then
f(S) ≤ f(T) and g(S) ≤ g(T ) for all S ⊆ T   implies   E(fg) ≥ (E f)(E g).

Here, as usual, E f stands for ΣS μ(S)f(S). The conclusion can also be written ‘covar(f, g) ≥ 0’, using the notation of (9); we say that f and g are “positively correlated” when this is true. (The awkward term “nonnegatively correlated” would be more accurate, because f and g might actually be independent.) Hint: Prove the result first in the special case that both f and g are nonnegative.

Furthermore,
f(S) ≥ f(T) and g(S) ≥ g(T ) for all S ⊆ T   implies   E(fg) ≥ (E f)(E g);
f(S) ≤ f(T) and g(S) ≥ g(T ) for all S ⊆ T   implies   E(fg) ≤ (E f)(E g).

It isn’t necessary to verify condition (**) for all sets, if (**) is known to hold for sufficiently many pairs of “neighboring” sets. Given μ, let’s say that set S is supported if μ(S) ≠ 0. Prove that (**) holds for all S and T whenever the following three conditions are satisfied: (i) If S and T are supported, so are S ∪ T and S ∩ T. (ii) If S and T are supported and S ⊆ T , the elements of T \ S can be labeled t_1, ... , tk such that each of the intermediate sets S ∪{t_1,...,tj} is supported, for 1 ≤ j ≤ k. (iii) Condition (**) holds whenever S = R ∪ s and T = R ∪ t and s, t / ∈R.

The multivariate Bernoulli distribution B(p_1,...,pm) on subsets of {1,...,m} is
μ(S)=(Wj=1mpj[j∈S])(Wj=1m(1−pj)[j∉S]),
given 0 ≤ p_1,...,pm ≤ 1. (Thus each element j is included independently with probability pj, as in exercise 25.) Show that this distribution satisfies (**).

Describe other simple distributions for which (**) holds.

62. [M20] Suppose the m=(n2) edges E of a random graph G on n vertices are chosen with the Bernoulli distribution B(p_1,...,pm). Let f(E) = [G is connected] and g(E) = [G is 4-colorable]. Prove that f is negatively correlated with g.

63. [M17] Suppose Z_0 and Z_1 are random ternary variables with Pr(Z_0 = a and Z_1 = b) = pab for 0 ≤ a, b ≤ 2, where p_00 + p_01 + ··· + p_22 = 1. What can you say about those nine probabilities pab when E(Z_1 | Z_0) = Z_0?

▶64. [M22] (a) If E(Zn+1 | Zn) = Zn for all n ≥ 0, is 〈Zn〉 a martingale? (b) If 〈Zn〉 is a martingale, is E(Zn+1 | Zn) = Zn for all n ≥ 0?

65. [M21] If 〈Zn〉 is any martingale, show that any subsequence 〈Zm(n)〉 is also a martingale, where the nonnegative integers 〈m(n)〉 satisfy m(0) < m(1) < m(2) < ···.

▶66. [M22] Find all martingales Z_0, Z_1, ... such that each random variable Zn assumes only the values ±n.

67. [M20] The Equitable Bank of El Dorado features a money machine such that, if you insert k dollars, you receive 2k dollars back with probability exactly 1/2; otherwise you get nothing. Thus you either gain $k or lose $k, and your expected profit is $0. (Of course these transactions are all done electronically.)

1. Consider, however, the following scheme: Insert $1; if that loses, insert $2; if that also loses, insert $4; then $8, etc. If you first succeed after inserting 2n dollars, stop (and take the 2n+1 dollars). What’s your expected net profit at the end?

2. Continuing (a), what’s the expected total amount that you put into the machine?

3. If Zn is your net profit after n trials, show that 〈Zn〉 is a martingale.

68. [HM23] When J. H. Quick (a student) visited El Dorado, he decided to proceed by making repeated bets of $1 each, and to stop when he first came out ahead. (He was in no hurry, and was well aware of the perils of the high-stakes strategy in exercise 67.)

1. What martingale 〈Zn〉 corresponds to this more conservative strategy?

2. Let N be the number of bets that Quick made before stopping. What is the probability that N = n?

3. What is the probability that N ≥ n?

4. What is E N?

5. What is the probability that min(Z_0,Z_1,...) = −m? (Possible “gambler’s ruin.”)

6. What is the expected number of indices n such that Zn = −m, given m ≥ 0?

69. [M20] Section 1.2.5 discusses two basic ways by which we can go from permutations of {1,...,n − 1} to permutations of {1,...,n}: “Method 1” inserts n among the previous elements in all possible ways; “Method 2” puts a number k from 1 to n in the final position, and adds 1 to each previous number that was ≥ k.

Show that, using either method, every permutation can be associated with a node of Fig. P, using a rule that obeys the probability assumptions of Pólya’s urn model.

70. [M25] If Pólya’s urn model is generalized so that we start with c balls of different colors, is there a martingale that generalizes Fig. P?

71. [M21] (G. Pólya.) What is the probability of going from node (r, b)tonode (r′,b′) in Fig. P, given r, r′, b, and b′ with r′ ≥ r and b′ ≥ b?

72. [M23] Let Xn be the red-ball indicator for Pólya’s urn, as discussed in the text. What is E(Xn1 Xn_2 ...Xnm) when 0 < n_1 < n_2 < ··· < nm?

73. [M24] The ratio Zn = r/(n+2)atnode (r, n+2−r) of Fig. P is not the only martingale definable on Pólya’s urn. For example, r[n = r − 1] is another; so is r(n+1r)/2n.

Find the most general martingale 〈Zn〉 for this model: Given any sequence a_0, a_1, ... , show that there’s exactly one suitable function Zn = f(r, n) such that f(1,k) = ak.

74. [M20] (Bernard Friedman’s urn.) Instead of contributing a ball of the same color, as in Fig. P, suppose we use the opposite color. Then the process changes to

and the probabilities of reaching each node become quite different. What are they?

75. [M25] Find an interesting martingale for Bernard Friedman’s urn.

76. [M20] If 〈Zn〉 and 〈Zn′〉 are martingales, is 〈Zn+Zn′〉 a martingale?

77. [M21] Prove or disprove: If 〈Zn〉 is a martingale with respect to 〈Xn〉 , then 〈Zn〉 is a martingale with respect to itself (that is, a martingale).

78. [M20] A sequence of random variables 〈Vn〉 for which E(Vn+1 | V_0,...,Vn) = 1 is called “multiplicatively fair.” Show that Zn = V_0V_1 ...Vn is a martingale in such a case. Conversely, does every martingale lead to a multiplicatively fair sequence?

79. [M20] (De Moivre’s martingale.) Let X_1, X_2, ... be a sequence of independent coin tosses, with Pr([“heads” occurred on the nth toss]) = Pr(Xn = 1) = p for each n. Show that Zn = (q/p)2(X_1+···+Xn)−n defines a martingale, where q = 1 − p.

80. [M20] Are the following statements true or false for every fair sequence 〈Yn〉? (a) E(Y32⁢ Y5) = 0. (b) E(Y3Y52⁢ ) = 0. (c) E(Yn_1Yn_2 ...Ynm) = 0 if n_1 < n_2 < ··· < nm.

81. [M21] Suppose E(Xn+1 | X_0,...,Xn) = Xn + Xn−1 for n ≥ 0, where X−1 = 0. Find sequences an and bn of coefficients so that Zn = anXn + bnXn−1 is a martingale, where Z_0 = X_0 and Z_1 = 2X_0 − X_1. (We might call this a “Fibonacci martingale.”)

▶82. [M20] In the game of Ace Now, let Xn = [the nth card is an ace], with X_0 = 0.

1. Show that Zn = (4 − X_1 −···− Xn)/(52 − n) satisfies (28) for 0 ≤ n < 52.

2. Consequently E ZN = 1/13, regardless of the stopping rule employed.

3. Hence all strategies are equally good (or bad); you win $0 on average.

▶83. [HM22] Given a sequence 〈Xn〉 of independent and nonnegative random variables, let Sn = X_1 + ··· + Xn. If Nn(x_0,...,xn−1) is any stopping rule and if N is defined by (31), prove that E ∑k=1NE Xk. (In particular, if E Xn = E X_1 for all n > 0 we have “Wald’s equation,” which states that E SN = (E N)(E X_1).)

84. [HM21] Let f(x) be a convex function for a ≤ x ≤ b, and assume that 〈Zn〉 is a martingale such that a ≤ Zn ≤ b for all n ≥ 0. (Possibly a = −∞ and/or b = + ∞.)

1. Prove that 〈f(Zn)〉 is a submartingale.

2. What can you say if the sequence 〈Zn〉 is assumed only to be a submartingale?

85. [M20] Suppose there are Rn red balls and Bn black balls at level n of Pólya’s urn (Fig. P). Prove that the sequence 〈Rn /Bn〉 is a submartingale.

▶86. [M22] Prove (33) by inventing a suitable stopping rule Nn+1(Z_0,...,Zn).

87. [M18] What does the maximal inequality (33) reveal about the chances that Pólya’s urn will hold thrice as many red balls as black balls at some point?

▶ 88. [HM30] Let S = sup Zn be the least upper bound of Zn as n → ∞ in Fig. P.

1. Prove that S > 1/2 with probability ln 2 ≈ .693.

2. Similarly, show that Pr(S > 2/3) = ln 3 − π/27 ≈ .494.

3. Generalize to Pr(S > (t − 1)/t), for all t ≥ 2. Hint: See exercise 7.2.1.6–36.

89. [M17] Let (X_1,...,Xn) be random variables that have the Bernoulli distribution B(p_1,...,pn), and suppose c_1, ... , cn are nonnegative. Use (37) to show that

Pr(c1X1+⋅⋅⋅+cnXn≥c1p1+⋅⋅⋅+cnpn+x)≤e−2x2/(c12+⋅⋅⋅+cn2).

90. [HM25] The Hoeffding–Azuma inequality (37) can be derived as follows:

1. Show first that Pr(Y_1 + ··· + Yn ≥ x) ≤ E(e(Y_1 + ··· + Yn)t)/etx for all t > 0.

2. If 0 ≤ p ≤ 1 and q = 1 − p, show that eyt ≤ ef(t) + yeg(t) when −p ≤ y ≤ q and t > 0, where f(t) = −pt + ln(q + pet) and g(t) = −pt + ln(et − 1).

3. Prove that f(t) ≤ t^2/8. Hint: Use Taylor’s formula, Eq. 1.2.11.3–(5).

4. Consequently a ≤ Y ≤ b implies eYt ≤ e(b−a)2t^2/8 + Yh(t), for some function h(t).

5. Let c=(c12+⋅⋅⋅+cn2)/2, where ck = bk − ak. Prove that E(e(Y_1+···+Yn)t) ≤ ect^2/4.

6. We obtain (37) by choosing the best value of t.

91. [M20] Prove that Doob’s general formula (39) always defines a martingale.

▶92. [M20] Let 〈Qn〉 be the Doob martingale that corresponds to Pólya’s urn (27) when Q = Xm, for some fixed m > 0. Calculate Q_0, Q_1, Q_2, etc.

93. [M20] Solve the text’s hashing problem under the more general model considered in the bin-packing problem: Each variable Xn has probability pnk of being equal to k, for 1 ≤ n ≤ t and 1 ≤ k ≤ m. What formula do you get instead of (44)?

▶94. [M22] Where is the fact that the variables {X_1,...,Xt} are independent used in the previous exercise?

95. [M20] True or false: “Pólya’s urn q.s. accumulates more than 100 red balls.”

96. [HM22] Let X be the number of heads seen in n flips of an unbiased coin. Decide whether each of the following statements about X is a.s., q.s., or neither, as n → ∞:

1. |X−n/2|<nln n;

2. |X−n/2|<n⁢ ln  n;

3. |X−n/2|<n⁢ ln  ln  n;

4. |X−n/2|<n⁢ .

▶ 97. [HM21] Suppose ⌊n1+δ⌋ items are hashed into n bins, where δ is a positive constant. Prove that every bin q.s. gets between 12nδ and 2nδ of them.

98. [M21] Many algorithms are governed by a loop of the form

X ← n; while X > 0, set X ← X − F (X)

where F (X) is a random integer in the range [1 ..X]. We assume that each integer F(X) is completely independent of any previously generated values, subject only to the requirement that E F (j) ≥ gj, where 0 < g_1 ≤ g_2 ≤ ··· ≤ gn.

Prove that the loop sets X ← X − F (X) at most 1/g_1+1/g_2 + ··· +1/gn times, on the average. (“If one step reduces by gn, then perhaps (1/gn)th of a step reduces by 1.”)

99. [HM30] Show that the result in the previous exercise holds even when the range of F (X) is (−∞ ..X], given 0 < g_1 ≤ ··· ≤ gn ≤ gn+1 ≤ ··· . (Thus X might increase.)

100. [HM17] A certain randomized algorithm takes T steps, where Pr(T = t) = pt for 1 ≤ t ≤ ∞. Prove that (a) limm→∞ E min(m, T ) = E T ; (b) E T < ∞ implies p∞ = 0.

101. [HM22] Suppose X = X_1 + ··· + Xm is the sum of independent geometrically distributed random integers, with Pr(Xk = n) = pk(1 − pk)n−1 for n ≥ 1. Prove that Pr(X ≥ rμ) ≤ re1−r for all r ≥ 1, where μ=E X⁢=∑k=1m1/pk.

102. [M20] Cora collects coupons, using a random process. After already owning k − 1 of them, her chance of success when trying for the kth is at least one chance in sk, independent of any previous successes or failures. Prove that she will a.s. own m coupons before making (s_1 + ··· + sm)ln n trials. And she will q.s. need at most sk ln n ln ln n trials to obtain the kth coupon, for each k ≤ m, if m = O(n^1000).

▶103. [M30] This exercise is based on two functions of the ternary digits {0, 1, 2}:

f0(x)=max(0,x−1);f1(x)=min(2,x+1).

What is Pr(fX_1(fX_2(... (fXn(i)) ...)) = j), for each i, j ∈ {0, 1, 2}, assuming that X_1, X_2, ... , Xn are independent, uniformly random bits?

Here’s an algorithm that computes fX_1(fX_2(... (fXn(i)) ...)) for i ∈ {0, 1, 2}, and stops when all three values have coalesced to a common value:

Set a_0a_1a_2 ← 012 and n ← 0. Then while a_0 = a_2, set n ← n + 1, t_0t_1t_2 ← (Xn? 122: 001), and a_0a_1a_2 ← at_0at_1at_2. Output a_0.

(Notice that a_0 ≤ a_1 ≤ a_2 always holds.) What is the probability that this algorithm outputs j? What are the mean and variance of N, the final value of n?

A similar algorithm computes fXn(... (fX_2(fX_1(i))) ...), if we change ‘at_0at_1at_2’ to ‘ta_0ta_1 ta_2’. What’s the probability of output j in this algorithm?

Why on earth are the results of (b) and (c) so different?

The algorithm in (c) doesn’t really use a_1. Therefore we might try to speed up process (b) by cleverly evaluating the functions in the opposite direction. Consider the following subroutine, called sub(T):

Set a_0a_2 ← 02 and n ← 0. Then while n < T set n ← n +1, X ← random bit, and a_0a_2 ← (Xn? f_1(a_0)f_1(a_2): f_0(a_0)f_0(a_2)). If a_0 = a_2 output a_0, otherwise output −1.

Then the algorithm of (b) would seem to be equivalent to

Set T ← 1, a ← −1; while a < 0 set T ← 2T and a ← sub(T ); output a.

Prove, however, that this fails. (Randomized algorithms can be quite delicate!)

Patch the algorithm of (e) and obtain a correct alternative to (b).

104. [M21] Solve exercise 103(b) and 103(c) when each Xk is 1 with probability p.

▶105.[M30] (Random walk on an n-cycle.) Given integers a and n, with 0 ≤ a ≤ n, let N be minimum such that (a +(−1)X_1 +(−1)X_2 + ··· +(−1)XN) mod n = 0, where X_1, X_2, ... is a sequence of independent random bits. Find the generating function ga=∑k=0∞Pr(N=k)zk. What are the mean and variance of N?

106. [M25] Consider the algorithm of exercise 103(b) when the digits are d-ary instead of ternary; thus f_0(x) = max(0,x − 1) and f_1(x) = min(d − 1, x + 1). Find the generating function, mean, and variance of the number N of steps required before a_0 = a_1 = ··· = ad−1 is first reached in this more general situation.

▶107. [M22] (Coupling.) If X is a random variable on the probability space Ω′ and Y is another random variable on another probability space Ω″, we can study them together by redefining them on a common probability space Ω. All conclusions about X or Y are valid with respect to Ω, provided that we have Pr(X = x) = Pr′(X = x) and Pr(Y = y) = Pr″(Y = y) for all x and y.

Such “coupling” is obviously possible if we let Ω be the set Ω′ × Ω″ of pairs {ω′ω″ | ω′ ∈ Ω′ and ω″ ∈ Ω″}, and if we define Pr(ω′ω″) = Pr′(ω′)Pr″(ω″) for each pair of events. But coupling can also be achieved in many other ways.

For example, suppose Ω′ and Ω″ each contain only two events, {Q, K} and {♣, ♠}, with Pr′(Q) = p, Pr′(K) = 1 − p, Pr″(♣) = q, Pr″(♠) = 1 − q. We could couple them with a four-event space Ω = {Q♣, Q♠, K♣, K♠}, having Pr(Q♣) = pq, Pr(Q♠) = p(1 − q), Pr(K♣) = (1 − p)q, Pr(K♠) = (1 − p)(1 − q). But if p < q we could also get by with just three events, letting Pr(Q♣) = p, Pr(K♣) = q − p, Pr(K♠) = 1 − q. A similar scheme works when p > q, omitting K♣. And if p = q we need only two events, Q♣ and K♠.

1. Show that if Ω′ and Ω″ each have just three events, with respective probabilities {p_1,p_2,p_3} and {q_1,q_2,q_3}, they can always be coupled in a five-event space Ω.

2. Also, four events suffice if {p1,p2,p3}={112,512,612},{q1,q2,q3}={212,312,712}.

3. But some three-event distributions cannot be coupled with fewer than five.

108. [HM21] If X and Y are integer-valued random variables such that Pr′(X ≥ n) ≤ Pr″(Y ≥ n) for all integers n, find a way to couple them so that X ≤ Y always holds.

109. [M27] Suppose X and Y have values in a finite partially ordered set P, and that

Pr′(X ≻ a for some a ∈ A) ≤ Pr″(Y ≻ a for some a ∈ A),    for all A ⊆ P.

We will show that there’s a coupling in which X ≺ Y always holds.

1. Write out exactly what needs to be proved, in the simple case where P = {1, 2, 3} and the partial order has 1 ≺ 3, 2 ≺ 3. (Let pk = Pr′(X = k) and qk = Pr″(Y = k) for k ∈ P. When P = {1,...,n}, a coupling is an n×n matrix (pij) of nonnegative probabilities whose row sums are ∑j pij = pi and column sums are ∑i pij = qj.) Compare this to the result proved in the preceding exercise.

2. Prove that Pr′(X ≺ b for some b ∈ B) ≥ Pr″(Y ≺ b for some b ∈ B), for all B ⊆ P.

3. A coupling between n pairs of events can be viewed as a flow in a network that has 2n + 2 vertices {s, x_1,...,xn,y_1,...,yn,t}, where there are pi units of flow from s to xi, pij units of flow from xi to yj, and qj units of flow from yj to t. The “max-flow min-cut theorem” [see Section 7.5.3] states that such a flow is possible if and only if there are no subsets I, J ⊆ {1,...,n} such that (i) every path from s to t goes through some arc s → xi for i ∈ I or some arc yj → t for j ∈ J, and (ii) ∑i∈I pi + ∑j∈J qj < 1. Use that theorem to prove the desired result.

110. [M25] If X and Y take values in {1,...,n}, let pk = Pr′(X = k), qk = Pr″(Y = k), and rk = min(pk,qk) for 1 ≤ k ≤ n. The probability that X = Y in any coupling is obviously at most r=∑k=1nrk.

1. Show that there always is a coupling with Pr(X = Y ) = r.

2. Can the result of the previous exercise be extended, so that we have not only Pr(X ≺ Y ) = 1 but also Pr(X = Y ) = r?

▶111. [M20] A family of N permutations of the numbers {1,...,n} is called minwise independent if, whenever 1 ≤ j ≤ k ≤ n and {a_1,...,ak} ⊆ {1,...,n}, exactly N/k of the permutations π have min(a_1π,...,akπ) = ajπ.

For example, the family F of N = 60 permutations obtained by cyclic shifts of

123456, 126345, 152346, 152634, 164235, 154263, 165324, 164523, 156342, 165432
can be shown to be minwise independent permutations of {1, 2, 3, 4, 5, 6}.

1. Verify the independence condition for F in the case k = 3, a_1 = 1, a_2 = 3, a_3 = 4.

2. Suppose we choose a random π from a minwise independent family, and assign the “sketch” SA = mina∈A aπ to every A ⊆ {1,...,n}. Prove that, if A and B are arbitrary subsets, Pr(SA = SB) = |A ∩ B| / |A ∪ B|.

3. Given three subsets A, B, C, what is Pr(SA = SB = SC)?

112. [M25] The size of a family F of minwise independent permutations must be a multiple of k for each k ≤ n, by definition. In this exercise we’ll see how to construct such a family with the minimum possible size, namely N = lcm(1, 2,...,n).

The basic idea is that, if all elements of the permutations in F that exceed m are replaced by ∞, the “truncated” family is still minwise independent in the sense that, if mina∈A aπ = ∞, we can imagine that the minimum occurs at a random element of A. (This can happen only if π takes all elements of A to ∞.)

1. Conversely, show that an m-truncated family can be lifted to an (m+1)-truncated family if, for each subset B of size n − m, we insert m + 1 equally often into each of B’s n − m positions, within the permutations whose ∞’s are in B.

2. Use this principle to construct minimum-size families F.

113. [M25] Although minwise permutations are defined only in terms of the minimum operation, a minwise independent family actually turns out to be also maxwise independent — and even more is true!

1. Let E be the event that aiπ < k, bπ = k, and cjπ > k, for any disjoint sets {a_1,...,al}, {b}, {c_1,...,cr} ⊆ {1,...,n}. Prove that, if π is chosen randomly from a minwise independent set, Pr(E) is the same as the probability that E occurs when π is chosen randomly from the set of all permutations. (For example, Pr(5π<7, 2π=7, 1π>7, 8π>7) = 6(n − 7)(n − 8)(n − 4)!/n!, whenever n ≥ 8.)

2. Furthermore, if {a_1,...,ak} ⊆ {1,...,n}, the probability that ajπ is the rth largest element of {a_1 π,...,akπ} is 1/k, whenever 1 ≤ j, r ≤ k.

▶114. [M28] (The “combinatorial nullstellensatz.”) Let f(x_1,...,xn) be a polynomial in which the coefficient of x1d1.⁢.⁢.⁢   xndn is nonzero and each term has degree ≤ d_1+···+dn. Given subsets S_1, ... , Sn of the field of coefficients, with |Sj| > dj for 1 ≤ j ≤ n, choose X_1, ... , Xn independently and uniformly, with each Xj ∈ Sj. Prove that

Pr(f(X1,.⁢ .⁢ .⁢,X2)≠0)≥|S1|+⋅⋅⋅+|Sn|−(d1+n)+1|S1|⋅⋅⋅|Sn|.

Hint: See exercise 4.6.1–16.

115. [M21] Prove that an m × n grid cannot be fully covered by p horizontal lines, q vertical lines, r diagonal lines of slope +1, and r diagonal lines of slope −1, if m = p + 2 ⌊r/2⌋ + 1 and n = q + 2 ⌈r/2⌉ + 1. Hint: Apply exercise 114 to a suitable polynomial f(x, y).

116. [HM25] Use exercise 114 to prove that, if p is prime, any multigraph G on n vertices with more than (p − 1)n edges contains a nonempty subgraph in which the degree of every vertex is a multiple of p. (In particular, if each vertex of G has fewer than 2p neighbors, G contains a p-regular subgraph. A loop from v to itself adds two to v’s degree.) Hint: Let the polynomial contain a variable xe for each edge e of G.

▶117. [HM25] Let X have the binomial distribution Bn(p), so that Pr(X=k) = (nk)pk(1−p)n−k for 0 ≤ k ≤ n. Prove that X mod m is approximately uniform:

|Pr(X⁢ mod m=r)−1m|<2m∑j=1∞e−8p(1−p)j2n/m2,          for 0≤r<m.

118. [M20] Use the second moment principle to prove the Paley–Zygmund inequality

Pr(X≥x) ≥(E X−x)2E X2,if 0 ≤x≤E X.

119. [HM24] Let x be a fixed value in [0 . . 1]. Prove that, if we independently and uniformly choose U ⋵ [0 . .x], V ⋵ [x . . 1], W ⋵ [0 . . 1], then the median 〈UVW〉 is uniformly distributed in [min(U, V,W) . . max(U, V,W)].

120. [M20] Consider random binary search trees Tn obtained by successively inserting independent uniform deviates U_1, U_2, . . . into an initially empty tree. Let Tnk be the number of external nodes on level k, and define Tn(z)=∑k=0∞Tnk⁢ zk/(n+1). Prove that Zn = Tn(z)/gn+1(z) is a martingale, where gn(z) = (2z + n−2)(2z + n−3) . . . (2z)/n! is the generating function for the cost of the nth insertion (exercise 6.2.2.6).

▶121. [M26] Let X and Y be random variables with the distributions Pr(X = t) = x(t) and Pr(Y = t) = y(t). The ratio ⍴(t) = y(t)/x(t), which may be infinity, is called the probability density of Y with respect to X. We define the relative entropy of X with respect to Y, also called the Kullback–Leibler divergence of X from Y, by the formulas

D(y||x)=E(ρ(X) lg ρ(X)) = E lg ρ(Y)=∑ty(t)lgy(t)x(t),
with 0 lg 0 and 0 lg(0/0) understood to mean 0. It can be viewed intuitively as the number of bits of information that are lost when X is used to approximate Y.

1. Suppose X is a random six-sided die with the uniform distribution, but Y is a “loaded” die in which Pr(Y = ) = 15 Pr(Y = ) = 215, instead of 16. Compute D(y||x) and D(x||y).

2. Prove that D(y||x) ≥ 0. When is it zero?

3. If p = Pr(X ∈ T ) and q = Pr(Y ∈ T ), show that E (lg ρ(Y) | Y ∈ T) ≥ lg(q/p).

4. Suppose x(t) = 1/m for all t in an m-element set S, and y(t) = 0 only when t ∈ S. Express D(y||x) in terms of the entropy HY = E lg(1/Y) (see Eq. 6.2.2–(18)).

5. Let Z(u, v) = Pr(X = u and Y = v) when X and Y have any joint distribution, and let W (u, v) be that same probability under the assumption that X and Y are independent. The joint entropy HX,Y is defined to be HZ, and the mutual information IX,Y is defined to be D(z||w). Prove that HW = HX + HY and IX,Y = HW − HZ. (Consequently HX,Y ≤ HX + HY , and IX,Y measures the difference.)

6. Let HX|Y = HX − IX,Y = HX,Y − HY = ∑t y(t)HX|t be the average uncertainty of X, in bits, after Y has been revealed. Prove that HX|(Y,Z) ≤ HX|Y .

122. [HM24] Continuing exercise 121, compute D(y||x) and D(x||y) when

1. x(t) = 1/2t+1 and y(t) = 3t/4t+1 for t = 0, 1, 2, ... ;

2. x(t) = e−np(np)t/t! and y(t)=(nt)pt(1−p)n−t, for t ≥ 0 and 0 < p < 1. (Give asymptotic answers with absolute error O(1/n), for fixed p as n → ∞.)

▶123. [M20] Let X and Y be as in exercise 121. The random variable Z = A? Y : X either has the distribution x(t) or y(t), but we don’t know whether A is true or false. If we believe that the hypothesis Z = Y holds with the a priori probability Pr(A) = pk, we assume that zk(t) = Prk(Z = t) = pkx(t) + (1 − pk)y(t). But after seeing a new value of Z, say Z = Zk, we will believe the hypothesis with the a posteriori probability pk+1 = Pr(A | Zk). Show that D(y||x) is the expected “information gained,” lg(pk+1/(1 − pk+1)) − lg(pk/(1 − pk)), averaged with respect to the distribution of Y.

124. [HM22] (Importance sampling.) In the setting of exercise 121, we have E f(Y ) = E(ρ(X)f(X)) for any function f; thus ρ(t) measures the “importance” of the X-value t with respect to the Y-value t. Many situations arise when it’s easy to generate random variables with an approximate distribution x(t), but difficult to generate them with the exact distribution y(t). In such cases we can estimate the average value E(f) = E f(Y ) by calculating En(f) = (ρ(X_1)f(X_1)+ ··· + ρ(Xn)f(Xn))/n, where the Xj are independent random variables, each distributed as x(t).

Let n = c^42D(y||x). Prove that if c > 1, this estimate En is relatively accurate:

|E(f)−En(f)|≤∥f∥(1/c+2Δc),Where Δc=Pr(ρ(Y)>c22D(y∥x)).

(Here ∥f∥ denotes (E f(Y)2)1/2.) On the other hand if c < 1 the estimate is poor:

Pr(En(1)≥a)≤c2+(1−Δc)/a.for 0 < a<1,

Here ‘1’ denotes the constant function f(y) = 1 (hence E(1) = 1).

▶125. [M28] Let 〈an〉 = a_0, a_1, a_2, ... be a sequence of nonnegative numbers with no “internal zeros” (no indices i < j < k such that ai > 0, aj = 0, ak > 0). We call it log-convex if an2 ≤ an−1an+1 for all n ≥ 1, and log-concave if an2 ≥ an−1an+1 for all n ≥ 1.

1. What sequences are both log-convex and log-concave?

2. If 〈an〉 is log-convex or log-concave, so is its “left shift” 〈an+1〉 = a_1, a_2, a_3, .... What can be said about the “right shift” 〈an−1〉 = c, a_0, a_1, ... , given c?

3. Show that a log-concave sequence has aman ≥ am−1an+1 whenever 1 ≤ m ≤ n.

4. If 〈an〉 and 〈bn〉 are log-convex, show that 〈an + bn〉 is also log-convex.

5. If 〈an〉 and 〈bn〉 are log-convex, show that 〈∑k(nk)akbn−k〉 is also log-concave?

6. If 〈an〉 and 〈bn〉 are log-concave, is 〈∑k akbn – k〉 also log-concave?

7. If 〈an〉 and 〈bn〉 are log-concave, is 〈∑k(nk)akbn−k〉 also log-concave?

126. [HM22] Suppose X_1, ..., Xn are independent binary random variables with E Xk = m/n for all k, where 0 ≤ m ≤ n. Prove that Pr(X_1+···+Xn = m) = Ω(n−1/2).

127. [HM30] Say that a binary vector x = x_1 ... xn is sparse if νx ≤ θn, where θ is a given threshold parameter, 0 < θ < 12. Let S(n, θ) be the number of sparse vectors.

1. Show that S(n, θ) ≤ 2H(θ)n, where H denotes entropy.

2. On the other hand, S(n, θ) is also Ω(2H(0)n/n).

3. Let X′ and X″ be independent and uniformly distributed sparse vectors, and let x be any binary vector, all of length n. Prove that x ⊕ X′ ⊕ X″ is q.s. not sparse. [Hint: Both X′ and X″ q.s. have nearly θn 1s. Furthermore exercise 126 can be used to pretend that the individual bits of x ⊕ X′ ⊕ X″ are independent.]

▶128. [HM26] Consider n independent processors that are competing for access to a shared database. They’re totally unable to communicate with each other, so they agree to adopt the following protocol: During each unit of time, called a “round,” each processor independently generates a random uniform deviate U and “pings” the database (attempts an access) if U < 1/n. If exactly one processor pings, its attempt succeeds; otherwise nobody gets access during that round.

1. What is the probability that some processor pings successfully, in a given round?

2. How many rounds does a particular processor have to wait, on average, before being successful? (Give an asymptotic answer, correct to O(1/n).)

3. Let ∈ be any positive constant. Prove that the processors a.s. will all have at least one success during the first (1 + ∈)en ln n rounds. Hint: See exercise 3.3.2–10.

4. But prove also that they a.s. will not all succeed during (1 − ∈)en ln n rounds.

129. [HM28] (General rational summation.) Let r(x) = p(x)/q(x), where p and q are polynomials, deg(q) ≥ deg(p) + 2, and q has no integer roots. Prove that

Σk=−∞∞p(k)q(k)=−πΣj=1t(Residue of r(z)cot πz⁢ ⁢at zj),
where z_1, ... , zt are the roots of q. Hint: Show that 12πi∮r(z)cot πz⁢dz=O(1/M), when the integral is taken along the square path for which max (|ℜz|,|ℑz|)=M+12.

Use this method to evaluate the following sums in “closed form”:

Σk=−∞∞1(2k−1)2;Σk=−∞∞1k2+1;Σk=−∞∞1k2+k+1;Σk=−∞∞1(k2+k+1)(2k−1).

130. [HM30] Many of the probability distributions that arise in modern computer applications have “heavy tails,” in contrast to bell-shaped curves that are concentrated near the mean. The simplest and most useful example — although it also is somewhat paradoxical — is the Cauchy distribution, defined by

Pr(X≤x)=1π∫−∞xdt1+t2.

1. If X is a Cauchy deviate, what are E X and E X^2?

2. What are Pr(|X| ≤ 1), Pr(|X|≤3), and Pr(|X|≤2+3)?

3. If U is a uniform deviate, show that tan(π(U − 1/2)) is a Cauchy deviate.

4. Suggest other ways to generate Cauchy deviates.

5. Let Z = pX +qY where X and Y are independent Cauchy deviates and p+q = 1, p, q > 0. Prove that Z has the Cauchy distribution.

6. Let X = (X_1,...,Xn) be a vector of n independent Cauchy deviates, and let c = (c_1,...,cn) be any vector of real numbers. What is the distribution of the dot product c · X = (c_1X_1 + ··· + cnXn)?

7. What is the “characteristic function” E eitX, when X is a Cauchy deviate?

131. [HM30] An integer-valued analog of Cauchy deviates, which we shall call the “iCauchy distribution” for convenience, has Pr(X = n) = c/(1 + n^2) for −∞ < n < ∞.

1. What constant c makes this a valid probability distribution?

2. Compare the distribution of X + Y to the distribution of 2Z, when X, Y , and Z are independent iCauchy deviates.

▸ 132. [HM26] Choose n balls from an urn that contains N balls, K of which are green.

1. What’s the probability pk that exactly k green balls are chosen?

2. What are the mean, modes, and variance? (A mode in a probability distribution is a value of k that’s a local maximum: pk−1 ≤ pk ≥ pk+1 and pk > 0.)

3. Let Xj = [the jth ball is green], so that pk = Pr(X_1 + ··· + Xn = k). Use a Doob martingale to establish an exponentially small upper bound on the tail probability Pr(X_1 + ··· + Xn ≥ nK/N + x).

133. [M25] Call t rows of a binary matrix shattered if all 2t possible columns occur.

1. Prove that any binary matrix with m rows and more than f(m,t)=(m0)+(m1)+...+(mt−1) distinct columns contains t shattered rows.

2. Construct a matrix with m rows and f(m, t) distinct columns, no t shattered.

134. [HM28] (V. N. Vapnik and A. Ya. Chervonenkis, 1971.) Many different events 𝓐 = {A_1,...,An}, which depend on each other in complicated ways, might be of interest simultaneously, and we often want to learn their probabilities pj = Pr(Aj) by observing a sufficiently large sample. If χ = {X_1,...,Xm} is a subset of the probability space Ω, the probability of sampling χ (with replacement) is Pr(X_1 ... Pr(Xm).

Consider the random m × n binary matrix whose entries are Xij = [Xi ∈ Aj] = [the atomic event Xi is an instance of Aj ]. The empirical probability P^j(χ) based on sample χ is then Mj (χ )/m, where Mj(χ ) = X_1j + ··· + Xmj, for 1 ≤ j ≤ n.

Let Ej(χ)=|P^j(χ)−pj| be the difference between the empirical and actual probabilities. We hope that the uniform sampling error E(χ) = max1≤j≤n Ej(χ) is small.

1. For all ∈ > 0 and 1 ≤ j ≤ n, prove that Pr(Ej(χ) > ∈) ≤ 1/(4∈2m).

2. Given independent m-samples χ and χ′, let E^j(χ,χ′)=|P^j(χ)−P^j(χ′)|. Show that Pr(E^j(χ,χ′)>∈)<2e−2∈2m. Hint: See exercise 132.

3. Let Δm(𝓐) be the maximum number of distinct columns that can appear in any of the m × n binary matrices obtainable from samples χ of size m. If m ≥ 2/∈2, use (a) and (b) to prove that Pr(E(χ)>∈)≤4Δ2m(A)e−∈2m/8.

[Note: The maximum d such that d atomic events of Ω can be shattered by the events of 𝓐 is called the Vapnik–Chervonenkis dimension of 𝓐. Exercise 133 shows that Δm(𝓐) has polynomial growth of degree d.]

135. [HM30] (Baxter permutations.) Let P = p_1 ...pn be a permutation of {1,...,n}, and let P− = q_1 ...qn be its inverse. These permutations are called Baxter permutations if and only if there are no indices k and l with 0 < k,l < n such that either (pk < l and pk+1 > l and ql > k and ql+1 < k) or (qk < l and qk+1 > l and pl > k and pl+1 < k).

What’s a good way to count the number bn of n-element Baxter permutations?

136. [HM20] Let f(x)=[x> 0]x ln x be the fundamental convex function that underlies formulas for entropy. Prove or disprove: If 0≤x≤y ≤1 then |f(y)−f(x)|≤|f(y−x)|.

137. [HM31] The median of a real-valued random variable X is a value m for which Pr(X≤m)≥12 and Pr(X≥m)≥12. For example, if X is a binary random variable with E X = p, then 1 is a median ⇔p≥12; 0 is a median ⇔p≤12; and a value m between 0 and 1 is a median ⇔p=12. Let med X be the set of all X’s medians.

1. Show that med X is always a closed interval [m..m¯], for some real m≤m¯.

2. If m<m¯, then Pr(X≤m)=Pr(X≥m¯)=12. (Discretely, X is never actually equal to any value of med X except for the two extreme elements m and m¯.)

3. True or false: If Pr(X∈[x..y])≥12 then [x..y] ⊇ med X.

4. Assuming that E |X − c| exists for all c, show that E |X − m| = minc E |X − c| if and only if m ∈ med X.

5. True or false: If μ = E X and σ^2 = var X and m ∈ med X then |μ − m| ≤ σ.

6. Prove an analog of Jensen’s inequality: If f is convex for all real values of x, then f(med X) ≤ med f(X), assuming that we interpret this formula properly in cases when med X and/or med f(X) aren’t unique.

▸ 138. [M21] (Law of total variance.) The “truly marvelous identity” (12), which is often called the law of total expectation, has an even more marvelous counterpart:

var(X) = var(E(X | Y )) + E(var(X | Y )).

“The overall variance of a random variable X is the variance of its average plus the average of its variance, with respect to any other random variable Y .” Prove it.

▸ 139. [HM33] (Frank Spitzer, 1956.) A random walk is defined by S_0 = 0 and Sn = Sn−1 + Xn for n > 0, where the integer-valued random variables X_1, X_2, ... are independent and have the same distribution. Let Sn+=max(Sn,0), Sn−=max(+Sn,0), Rn = max(S_0,S_1,...,Sn, Rn+=Rn−Sn, and define the generating functions
rn(w,z)=Σj,kPr(Rn=j,Rn+=k)wjzk,Sn+(z)=ΣkPr(Sn+=k)zk,Sn−(z)=ΣkPr(Sn−=k)zk.

Prove that these three basic quantities are related by the remarkable formula

Σn=0∞rn(w,z)tn=exp(Σn=1∞(sn+(w)+sn−(z)−1)tnn).

▸ 140. [HM34] (Smoothed analysis.) Algorithms are traditionally analyzed by either studying their worst case or an “average” case. A nice compromise between these extremes was introduced by D. A. Spielman and S.-H. Teng in JACM 51 (2004), 385–463: An adversary sets up the data for some particular case, and this data is perturbed by some random process. Then we analyze the expected running time when the algorithm is applied to the perturbed data, maximizing over all cases.

The purpose of this exercise is to carry out a smoothed analysis of Algorithm 1.2.10M, the very first algorithm that was analyzed in TAOCP: Given a sequence X = x_1 ...xn of distinct numbers, let λ(X) be the number of left-to-right maxima, namely the number of indices k with xk > xj for 1 ≤ j < k. When X is a random permutation, we showed in Section 1.2.10 that E λ(X) = Hn ≈ ln n and varλ(X)=Hn−Hn(2). On the other hand, λ(X) can be as large as n.

Several natural models will give a smooth transition between ln n and n, when we suppose that an arbitrary sequence X¯=x¯1...x¯n is perturbed to get X = x_1 ...xn.

1. Given a permutation X¯ of {1,...,n}, mark each x¯k with probability p (independently); then permute the marked elements uniformly to get X. What is E λ(X) when X¯=12...n (the only case for which λ(X¯)=n), and 0 < p < 1 is fixed?

2. Continuing (a), explore E λ(X) when X¯=(n−m+1)...n1...(n−m),p=12.

3. Continuing (a) and (b), show that Eλ(X)=O((nlog n)/p) for all X¯.

4. A single swap in model (a) can reduce λ(X) from n to 1! So the following model is better: Let 0≤x¯k≤1 for 1 ≤ k ≤ n, and set xk←x¯k+δk, where δk is uniformly random in [−∈..∈]. Show that E λ(X) is greatest when x¯1≤⋅⋅⋅≤x¯n.

5. Continuing (d), show that in this model we have Eλ(X)=O(n/∈+logn).

141. [M20] (Arithmetic and geometric mean inequality). When xk,pk > 0, prove that
p1x1+p2x2+⋅⋅⋅+pnxnp1+p2+⋅⋅⋅+pn≥(x1p1x2p2...xnpn)1/(p1+p2+...+pn).

(For integer pk, these are the means of the multiset {p_1 · x_1,p_2 · x_2,...,pn · xn}.)

▸ 142. [M30] (L. J. Rogers, 1887.) Let Mr = E |X|r be the rth absolute “moment” of the random variable X. (In particular, Mr = ∞ if r < 0 and Pr(X =0) > 0.)

1. Suppose q ≤ r ≤ s ≤ t and q +t = r +s. Set the values of (aj, bj, xj, yj) in Binet’s identity, exercise 1.2.3–30, to (pj,pjxjs−q,xjq,xjr), where p_1, p_2, ... are probabilities that sum to 1. What inequality involving Mq, Mr, Ms, Mt do you get?

2. Deduce from exercise 141 that Mqs−rMrq−sMsr−q≥1 when q < r < s and Mr < ∞. Hint: What happens when pj and xj are replaced respectively by pjxjr and xjs−r?

3. Let p > 1. Use the fact that M1/p≤M11/pto prove Hölder’s inequality:

Σk=1nakbk≤(Σk=1nakp)1p(Σk=1nbkq)1q,if1p+1q=1and ak,bk≥0.

4. Consequently | E XY | ≤ (E |X|p)1/p(E |Y |q)1/q.

143. [M22] For p > 1, use Hölder’s inequality to prove Minkowski’s inequality:

(E |X + Y |p)1/p ≤ (E |X|p)1/p + (E |Y |p)1/p.

144. [HM26] If E X exists and is finite, clearly E(X − E X) = 0.

1. If p ≥ 1 and E Y = 0, then E |X|p ≤ E |X + Y |p when X and Y are independent.

2. The symmetrization of a random variable X is Xsym = X+ − X−, where X+ and X− are independent random variables, each with the same distribution as X. Prove that p ≥ 1 and E X = 0 implies E |X|p ≤ E |Xsym|p.

3. Suppose X_1, ... , Xn are independent random variables that are symmetric about 0, in the sense that Pr(Xj = x) = Pr(Xj = −x) for 1 ≤ j ≤ n and all x. Prove that E |X_1|p + ··· + E |Xn|p ≤ E |X_1 + ··· + Xn|p, when p ≥ 2. Hint: |x|p+|y|p≤12(|x+y|p+|x−y|p).

4. Now suppose only that X_1, ... , Xn are independent with E X_1 = ··· = E Xn = 0. Prove that E |X_1|p + ··· +E |Xn|p ≤ 2p E |X_1 + ··· + Xn|p for p ≥ 2.

▸ 145. [M20] (Khinchin’s inequality.) Let a_1, ... , an be real numbers and let X_1, ... , Xn be random signs: Independently, each Xk is equally likely to be +1 or −1. Prove that

(a12+⋅⋅⋅an2)m≤E((a1X1+⋅⋅⋅+anXn)2m)≤(2m−1)!!(a12+⋅⋅⋅+an2)m
for all integers m ≥ 0, where (2m−1)!!=Πk=1m(2k−1) is a “semifactorial.”

146. [M25] (Marcinkiewicz and Zygmund’s inequality.) Let X_1, ... , Xn be independent random variables, each with mean 0 but possibly with different distributions. Then

122mE((Σk=1nXk2)m)≤E((Σk=1nXk)2m)≤22m(2m−1)!!E((Σk=1nXk2)m).

147. [M34] (Rosenthal’s inequality.) Under the assumptions of exercise 146,

122mB≤E((Σk=1nXk)2m)≤2m2+2m(2m−1)!!B,B=max(Σk=1nEXk2m,(Σk=1nEXk2)m).

Every man must judge for himself between conflicting vague probabilities.

— CHARLES DARWIN, letter to N. A. von Mengden (5 June 1879)

Chapter 7—Combinatorial Searching

Nowhere to go but out,

Nowhere to come but back.

— BEN KING, in The Sum of Life (c. 1893)

Lewis back-tracked the original route up the Missouri.

— LEWIS R. FREEMAN, in National Geographic Magazine (1928)

When you come to one legal road that’s blocked,
you back up and try another.

— PERRY MASON, in The Case of the Black-Eyed Blonde (1944)

7.2.2. Backtrack Programming

Now that we know how to generate simple combinatorial patterns such as tuples, permutations, combinations, partitions, and trees, we’re ready to tackle more exotic patterns that have subtler and less uniform structure. Instances of almost any desired pattern can be generated systematically, at least in principle, if we organize the search carefully. Such a method was christened “backtrack” by R. J. Walker in the 1950s, because it is basically a way to examine all fruitful possibilities while exiting gracefully from situations that have been fully explored.

Most of the patterns we shall deal with can be cast in a simple, general framework: We seek all sequences x_1x_2 ...xn for which some property Pn(x_1,x_2,...,xn) holds, where each item xk belongs to some given domain Dk of integers. The backtrack method, in its most elementary form, involves the invention of intermediate “cutoff” properties Pl(x_1,...,xl) for 1 ≤ l < n, such that

Pl(x1,...,xl) is true whenever ⁢Pl+1(x1,...,xl+1) is true;⁢(1)

Pl(x1,...,xl) is fairly easy to test, if ⁢Pl−1(x1,...,xl−1) holds.⁢(2)

(We assume that P_0() is always true. Exercise 1 shows that all of the basic patterns studied in Section 7.2.1 can easily be formulated in terms of domains Dk and cutoff properties Pl.) Then we can proceed lexicographically as follows:

Algorithm B (Basic backtrack). Given domains Dk and properties Pl as above, this algorithm visits all sequences x_1x_2 ...xn that satisfy Pn(x_1,x_2,...,xn).

B1. [Initialize.] Set l ← 1, and initialize the data structures needed later.

B2. [Enter level l.] (Now Pl−1(x_1,...,xl−1) holds.) If l > n, visit x_1x_2 ...xn and go to B5. Otherwise set xl ← min Dl, the smallest element of Dl.

B3. [Try xl.] If Pl(x_1,...,xl) holds, update the data structures to facilitate testing Pl+1, set l ← l + 1, and go to B2.

B4. [Try again.] If xl ≠ max Dl, set xl to the next larger element of Dl and return to B3.

B5. [Backtrack.] Set l ← l−1. If l > 0, downdate the data structures by undoing the changes recently made in step B3, and return to B4. (Otherwise stop.)

The main point is that if Pl(x_1,...,xl) is false in step B3, we needn’t waste time trying to append any further values xl+1 ...xn. Thus we can often rule out huge regions of the space of all potential solutions. A second important point is that very little memory is needed, although there may be many, many solutions.

For example, let’s consider the classic problem of n queens: In how many ways can n queens be placed on an n × n board so that no two are in the same row, column, or diagonal? We can suppose that one queen is in each row, and that the queen in row k is in column xk, for 1 ≤ k ≤ n. Then each domain Dk is {1, 2,...,n}; and Pn(x_1,...,xn) is the condition that
xj⁢≠xk⁢⁢             and      |xk−xj|≠k−j,        for 1≤j<k≤n.⁢(3)

(If xj = xk and j < k, two queens are in the same column; if |xk − xj| = k − j, they’re in the same diagonal.)

This problem is easy to set up for Algorithm B, because we can let property Pl(x_1,...,xl) be the same as (3) but restricted to 1 ≤ j < k ≤ l. Condition (1) is clear; and so is condition (2), because Pl requires testing (3) only for k = l when Pl−1 is known. Notice that P_1(x_1) is always true in this example.

One of the best ways to learn about backtracking is to execute Algorithm B by hand in the special case n = 4 of the n queens problem: First we set x_1 ← 1. Then when l = 2 we find P_2(1, 1) and P_2(1, 2) false; hence we don’t get to l = 3 until trying x_2 ← 3. Then, however, we’re stuck, because P_3(1, 3,x) is false for 1 ≤ x ≤ 4. Backtracking to level 2, we now try x_2 ← 4; and this allows us to set x_3 ← 2. However, we’re stuck again, at level 4; and this time we must back up all the way to level 1, because there are no further valid choices at levels 3 and 2. The next choice x_1 ← 2 does, happily, lead to a solution without much further ado, namely x_1x_2x_3x_4 = 2413. And one more solution (3142) turns up before the algorithm terminates.

The behavior of Algorithm B is nicely visualized as a tree structure, called a search tree or backtrack tree. For example, the backtrack tree for the four queens problem has just 17 nodes,

corresponding to the 17 times step B2 is performed. Here xl is shown as the label of an edge from level l − 1 to level l of the tree. (Level l of the algorithm actually corresponds to the tree’s level l − 1, because we’ve chosen to represent patterns using subscripts from 1 to n instead of from 0 to n−1 in this discussion.) The profile (p_0,p_1,...,pn) of this particular tree — the number of nodes at each level — is (1, 4, 6, 4, 2); and we see that the number of solutions, pn = p_4, is 2.

Figure 68 shows the corresponding tree when n = 8. This tree has 2057 nodes, distributed according to the profile (1, 8, 42, 140, 344, 568, 550, 312, 92). Thus the early cutoffs facilitated by backtracking have allowed us to find all 92 solutions by examining only 0.01% of the 8^8 = 16,777,216 possible sequences x_1 ... x_8. (And 8^8 is only 0.38% of the (648)=4,426,165,368 ways to put eight queens on the board.)

Fig. 68. The problem of placing eight nonattacking queens has this backtrack tree.

Notice that, in this case, Algorithm B spends most of its time in the vicinity of level 5 below the root. Such behavior is typical: The tree for n = 16 queens has 1,141,190,303 nodes, and its profile is (1, 16, 210, 2236, 19688, 141812, 838816, 3998456, 15324708, 46358876, 108478966, 193892860, 260303408, 253897632, 171158018, 72002088, 14772512), concentrated near level 12.

Data structures

Backtrack programming is often used when a huge tree of possibilities needs to be examined. Thus we want to be able to test property Pl as quickly as possible in step B3.

One way to implement Algorithm B for the n queens problem is to avoid auxiliary data structures and simply to make a bunch of sequential comparisons in that step: “Is xl −xj ∈{j −l, 0,l−j} for some j < l?” Assuming that we must access memory whenever referring to xj, given a trial value xl in a register, such an implementation performs approximately 112 billion memory accesses when n = 16; that’s about 98 mems per node.

We can do better by introducing three simple arrays. Property Pl in (3) says essentially that the numbers xk are distinct, and so are the numbers xk + k, and so are the numbers xk − k. Therefore we can use auxiliary Boolean arrays a_1 ...an, b_1 ...b_2n−1, and c_1 ...c_2n−1, where aj means ‘some xk = j’, bj means ‘some xk + k − 1 = j’, and cj means ‘some xk − k + n = j’. Those arrays are readily updated and downdated if we customize Algorithm B as follows:

B1*. [Initialize.] Set a_1 ...an ← 0 ... 0, b_1 ...b_2n−1 ← 0 ... 0, c_1 ...c_2n−1 ← 0 ... 0, and l ← 1.

B2*. [Enter level l.] (Now Pl−1(x_1,...,xl−1) holds.) If l > n, visit x_1x_2 ...xn and go to B5*. Otherwise set t ← 1.

B3*. [Try t.] If at = 1 or bt+l−1 = 1 or ct−l+n = 1, go to B4*. Otherwise set at ← 1, bt+l−1 ← 1, ct−l+n ← 1, xl ← t, l ← l + 1, and go to B2*.

B4*. [Try again.] If t < n, set t ← t + 1 and return to B3*.

B5*. [Backtrack.] Set l ← l − 1. If l > 0, set t ← xl, ct−l+n ← 0, bt+l−1 ← 0, at ← 0, and return to B4*. (Otherwise stop.)

Notice how step B5* neatly undoes the updates that step B3* had made, in the reverse order. Reverse order for downdating is typical of backtrack algorithms, although there is some flexibility; we could, for example, have restored at before bt+l−1 and ct−l+n, because those arrays are independent.

The auxiliary arrays a, b, c make it easy to test property Pl at the beginning of step B3*, but we must also access memory when we update them and downdate them. Does that cost us more than it saves? Fortunately, no: The running time for n = 16 goes down to about 34 billion mems, roughly 30 mems per node.

Furthermore we could keep the bit vectors a, b, c entirely in registers, on a machine with 64-bit registers, assuming that n ≤ 32. Then there would be just two memory accesses per node, namely to store xl ← t and later to fetch t ← xl. However, quite a lot of in-register computation would become necessary.

Walker’s method

The 1950s-era programs of R. J. Walker organized backtracking in a somewhat different way. Instead of letting xl run through all elements of Dl, he calculated and stored the set

Sl←{x∈Dl| Pl(x1,...,xι−1,x) holds}(5)

upon entry to each node at level l. This computation can often be done efficiently all at once, instead of piecemeal, because some cutoff properties make it possible to combine steps that would otherwise have to be repeated for each x ∈ Dl. In essence, he used the following variant of Algorithm B:

Algorithm W (Walker’s backtrack). Given domains Dk and cutoffs Pl as above, this algorithm visits all sequences x_1x_2 ...xn that satisfy Pn(x_1,x_2,...,xn).

W1. [Initialize.] Set l ← 1, and initialize the data structures needed later.

W2. [Enter level l.] (Now Pl−1(x_1,...,xl−1) holds.) If l > n, visit x_1x_2 ...xn and go to W4. Otherwise determine the set Sl as in (5).

W3. [Try to advance.] If Sl is nonempty, set xl ← min Sl, update the data structures to facilitate computing Sl+1, set l ← l + 1, and go to W2.

W4. [Backtrack.] Set l ← l − 1. If l > 0, downdate the data structures by undoing changes made in step W3, set Sl ← Sl \ xl, and retreat to W3.

Walker applied this method to the n queens problem by computing Sl = U \ Al \ Bl \ Cl, where U = Dl = {1,...,n} and

Al={xj|1≤j<l}, Bl={xj+j−l|1≤j<l}, Cl={xj−j+l|1≤j<l}.(6)

He represented these auxiliary sets by bit vectors a, b, c, analogous to (but different from) the bit vectors of Algorithm B* above. Exercise 10 shows that the updating in step W3 is easy, using bitwise operations on n-bit numbers; furthermore, no downdating is needed in step W4. The corresponding run time when n = 16 turns out to be just 9.1 gigamems, or 8 mems per node.

Let Q(n) be the number of solutions to the n queens problem. Then we have

n

=

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

Q(n)

=

1

1

0

0

2

10

4

40

92

352

724

2680

14200

73712

365596

2279184

14772512

and the values for n ≤ 11 were computed independently by several people during the nineteenth century. Small cases were relatively easy; but when T. B. Sprague had finished computing Q(11) he remarked that “This was a very heavy piece of work, and occupied most of my leisure time for several months. ... It will, I imagine, be scarcely possible to obtain results for larger boards, unless a number of persons co-operate in the work.” [See Proc. Edinburgh Math. Soc. 17 (1899), 43–68; Sprague was the leading actuary of his day.] Nevertheless, H. Onnen went on to evaluate Q(12) = 14,200 — an astonishing feat of hand calculation — in 1910. [See W. Ahrens, Math. Unterhaltungen und Spiele 2, second edition (1918), 344.]

All of these hard-won results were confirmed in 1960 by R. J. Walker, using the SWAC computer at UCLA and the method of exercise 10. Walker also computed Q(13); but he couldn’t go any further with the machine available to him at the time. The next step, Q(14), was computed by Michael D. Kennedy at the University of Tennessee in 1963, commandeering an IBM 1620 for 120 hours. S. R. Bunch evaluated Q(15) in 1974 at the University of Illinois, using about two hours on an IBM System 360-75; then J. R. Bitner found Q(16) after about three hours on the same computer, but with an improved method.

Computers and algorithms have continued to get better, of course, and such results are now obtained almost instantly. Hence larger and larger values of n lie at the frontier. The whopping value Q(27) = 234,907,967,154,122,528, found in 2016 by Thomas B. Preußer and Matthias R. Engelhardt, probably won’t be exceeded for awhile! [See J. Signal Processing Systems 88 (2017), 185–201. This distributed computation occupied a dynamic cluster of diverse FPGA devices for 383 days; those devices provided a total peak of more than 7000 custom-designed hardware solvers to handle 2,024,110,796 independent subproblems.]

Permutations and Langford pairs

Every solution x_1 ...xn to the n queens problem is a permutation of {1,...,n}, and many other problems are permutation-based. Indeed, we’ve already seen Algorithm 7.2.1.2X, which is an elegant backtrack procedure specifically designed for special kinds of permutations. When that algorithm begins to choose the value of xl, it makes all of the appropriate elements {1, 2,...,n}\{x_1,...,xl−1} conveniently accessible in a linked list.

We can get further insight into such data structures by returning to the problem of Langford pairs, which was discussed at the very beginning of Chapter 7. That problem can be reformulated as the task of finding all permutations of {1, 2,...,n}∪{−1, −2,..., −n} with the property that
xj=k⁢         implies  xj+k+1=−k,                    for 1≤j≤2n⁢  and 1 ≤k≤n.(7)

For example, when n = 4 there are two solutions, namely 2342¯13¯1¯4¯ and 4131¯24¯3¯2¯. (As usual we find it convenient to write 1¯ for −1,2¯ for −2, etc.) Notice that if x = x_1x_2 ... x_2n is a solution, so is its “dual” −xR = (−x_2n) ... (−x_2)(−x_1).

Here’s a Langford-inspired adaptation of Algorithm 7.2.1.2X, with the former notation modified slightly to match Algorithms B and W: We want to maintain pointers p_0p_1 ...pn such that, if the positive integers not already present in x_1 ...xl−1 are k_1 < k_2 < ... < kt when we’re choosing xl, we have the linked list
p0=k1,pk1=k2,...,pkt−1=kt, pkt=0.(8)

Such a condition turns out to be easy to maintain.

Algorithm L (Langford pairs). This algorithm visits all solutions x_1 ... x_2n to (7) in lexicographic order, using pointers p_0p_1 ... pn that satisfy (8), and also using an auxiliary array y_1 ... y_2n for backtracking.

L1. [Initialize.] Set x_1 ... x_2n ← 0 ... 0, pk ← k +1 for 0 ≤ k < n, pn ← 0, l ← 1.

L2. [Enter level l.] Set k ← p_0. If k = 0, visit x_1x_2 ... x_2n and go to L5. Otherwise set j ← 0, and while xl < 0 set l ← l +1.

L3. [Try xl = k.] (At this point we have k = pj.) If l + k +1 > 2n, go to L5. Otherwise, if xl+k+1 = 0, set xl ← k, xl+k+1 ← –k, yl ← j, pj ← pk, l ← l + 1, and return to L2.

L4. [Try again.] (We’ve found all solutions that begin with x_1 ... xl–1k or something smaller.) Set j ← k and k ← pj, then go to L3 if k ≠ 0.

L5. [Backtrack.] Set l ← l – 1. If l > 0 do the following: While xl < 0, set l ← l – 1. Then set k ← xl, xl ← 0, xl+k+1 ← 0, j ← yl, pj ← k, and go back to L4. Otherwise terminate the algorithm.

Careful study of these steps will reveal how everything fits together nicely. Notice that, for example, step L3 removes k from the linked list (8) by simply setting pj ← pk. That step also sets xl+k+1 ← –k, in accordance with (7), so that we can skip over position l + k + 1 when we encounter it later in step L2.

The main point of Algorithm L is the somewhat subtle way in which step L5 undoes the deletion operation by setting pj ← k. The pointer pk still retains the appropriate link to the next element in the list, because pk has not been changed by any of the intervening updates. (Think about it.) This is the germ of an idea called “dancing links” that we will explore in Section 7.2.2.1.

To draw the search tree corresponding to a run of Algorithm L, we can label the edges with the positive choices of xl as we did in (4), while labeling the nodes with any previously set negative values that are passed over in step L2. For instance the tree for n = 4 is

Solutions appear at depth n in this tree, even though they involve 2n values x_1x_2 ... x_2n.

Algorithm L sometimes makes false starts and doesn’t realize the problem until probing further than necessary. Notice that the value xl = k can appear only when l + k + 1 ≤ 2n; hence if we haven’t seen k by the time l reaches 2n – k – 1, we’re forced to choose xl = k. For example, the branch 121¯ in (9) needn’t be pursued, because 4 must appear in {x_1,x_2,x_3}. Exercise 20 explains how to incorporate this cutoff principle into Algorithm L. When n = 17, it reduces the number of nodes in the search tree from 1.29 trillion to 330 billion, and reduces the running time from 25.0 teramems to 8.1 teramems. (The amount of work has gone up from 19.4 mems per node to 24.4 mems per node, because of the extra tests for cutoffs, yet there’s a significant overall reduction.)

Furthermore, we can “break the symmetry” by ensuring that we don’t consider both a solution and its dual. This idea, exploited in exercise 21, reduces the search tree to just 160 billion nodes and costs just 3.94 teramems — that’s 24.6 mems per node.

Word rectangles

Let’s look next at a problem where the search domains Dl are much larger. An m × n word rectangle is an array of n-letter words* whose columns are m-letter words. For example,

* Whenever five-letter words are used in the examples of this book, they’re taken from the 5757 Stanford GraphBase words as explained at the beginning of Chapter 7. Words of other lengths are taken from The Official SCRABBLE® Players Dictionary, fourth edition (Hasbro, 2005), because those words have been incorporated into many widely available computer games. Such words have been ranked according to the British National Corpus of 2007 — where ‘the’ occurs 5,405,633 times and the next-most common word, ‘of’, occurs roughly half as often (3,021,525). The OSPD4 list includes respectively (101, 1004, 4002, 8887, 15727, 23958, 29718, 29130, 22314, 16161, 11412) words of lengths (2, 3, ..., 12), of which (97, 771, 2451, 4474, 6910, 8852, 9205, 8225, 6626, 4642, 3061) occur at least six times in the British National Corpus.

STATUSLOWESTUTOPIAMAKINGSLEDGE(10)

is a 5 × 6 word rectangle whose columns all belong to WORDS(5757), the collection of 5-letter words in the Stanford GraphBase. To find such patterns, we can suppose that column l contains the xlth most common 5-letter word, where 1 ≤ xl ≤ 5757 for 1 ≤ l ≤ 6; hence there are 5757^6 = 36,406,369,848,837,732,146,649 ways to choose the columns. In (10) we have x_1 ... x_6 = 1446 185 1021 2537 66 255. Of course very few of those choices will yield suitable rows; but backtracking will hopefully help us to find all solutions in a reasonable amount of time.

We can set this problem up for Algorithm B by storing the n-letter words in a trie (see Section 6.3), with one trie node of size 26 for each l-letter prefix of a legitimate word, 0 ≤ l ≤ n.

For example, such a trie for n = 6 represents 15727 words with 23667 nodes. The prefix ST corresponds to node number 260, whose 26 entries are

(484,0,0,0,1589,0,0,0,2609,0,0,0,0,0,1280,0,0,251,0,0,563,0,0,0,1621,0);(11)

this means that STA is node 484, STE is node 1589, ..., STY is node 1621, and there are no 6-letter words beginning with STB, STC, ..., STX, STZ. A slightly different convention is used for prefixes of length n – 1; for example, the entries for node 580, ‘CORNE’, are

(3879,0,0,3878,0,0,0,0,0,0,0,9602,0,0,0,0,0,171,0,5013,0,0,0,0,0,0),(12)

meaning that CORNEA, CORNED, CORNEL, CORNER, and CORNET are ranked 3879, 3878, 9602, 171, and 5013 in the list of 6-letter words.

Suppose x_1 and x_2 specify the 5-letter column-words SLUMS and TOTAL as in (10). Then the trie tells us that the next column-word x_3 must have the form c_1c_2c_3c_4c_5 where c_1 ∈ {A, E, I, O, R, U, Y}, c_2 ∉ {E, H, J, K, Y, Z}, c_3 ∈ {E, M, O, T}, c_4 ∉ {A, B, O}, and c_5 ∈ {A, E, I, O, U, Y}. (There are 221 such words.)

Let al_1 ... alm be the trie nodes corresponding to the prefixes of the first l columns of a partial solution to the word rectangle problem. This auxiliary array enables Algorithm B to find all solutions, as explained in exercise 24. It turns out that there are exactly 625,415 valid 5 × 6 word rectangles, according to our conventions; and the method of exercise 24 needs about 19 teramems of computation to find them all. In fact, the profile of the search tree is

(1,5757; 2458830, 360728099, 579940198, 29621728, 625415),(13)

indicating for example that just 360,728,099 of the 5757^3 = 190,804,533,093 choices for x_1x_2x_3 will lead to valid prefixes of 6-letter words.

With care, exercise 24’s running time can be significantly decreased, once we realize that every node of the search tree for 1 ≤ l ≤ n requires testing 5757 possibilities for xl in step B3. If we build a more elaborate data structure for the 5-letter words, so that it becomes easy to run though all words that have a specific letter in a specific position, we can refine the algorithm so that the average number of possibilities per level that need to be investigated becomes only

(5757.0, 1697.9⁢ , 844.1, 273.5, 153.5, 100.8);(14)

the total running time then drops to 1.15 teramems. Exercise 25 has the details. And exercise 28 discusses a method that’s faster yet.

Commafree codes

Our next example deals entirely with four-letter words. But it’s not obscene; it’s an intriguing question of coding theory. The problem is to find a set of four-letter codewords that can be decoded even if we don’t put spaces or other delimiters between them. If we take any message that’s formed from strings of the set by simply concatenating them together, likethis, and if we look at any seven consecutive letters ... x_1x_2x_3x_4x_5x_6x_7 ..., exactly one of the four-letter substrings x_1x_2x_3x_4, x_2x_3x_4x_5, x_3x_4x_5x_6, x_4x_5x_6x_7 will be a codeword. Equivalently, if x_1x_2x_3x_4 and x_5x_6x_7x_8 are codewords, then x_2x_3x_4x_5 and x_3x_4x_5x_6 and x_4x_5x_6x_7 aren’t. (For example, iket isn’t.) Such a set is called a “commafree code” or a “self-synchronizing block code” of length four.

Commafree codes were introduced by F. H. C. Crick, J. S. Griffith, and L. E. Orgel [Proc. National Acad. Sci. 43 (1957), 416–421], and studied further by S. W. Golomb, B. Gordon, and L. R. Welch [Canadian Journal of Mathematics 10 (1958), 202–209], who considered the general case of m-letter alphabets and n-letter words. They constructed optimum commafree codes for all m when n = 2, 3, 5, 7, 9, 11, 13, and 15; and optimum codes for all m were subsequently found also for n = 17, 19, 21, ... (see exercise 37). We will focus our attention on the four-letter case here (n = 4), partly because that case is still very far from being resolved, but mostly because the task of finding such codes is especially instructive. Indeed, our discussion will lead us naturally to an understanding of several significant techniques that are important for backtrack programming in general.

To begin, we can see immediately that a commafree codeword cannot be “periodic,” like dodo or gaga. Such a string already appears within two adjacent copies of itself. Thus we’re restricted to aperiodic strings like item, of which there are m^4 – m^2. Notice further that if item has been chosen, we aren’t allowed to include any of its cyclic shifts temi, emit, or mite, because they all appear within itemitem. Hence the maximum number of codewords in our commafree code cannot exceed (m^4 – m^2)/4.

For example, consider the binary case, m = 2, when this maximum is 3. Can we choose three four-bit “words,” one from each of the cyclic classes

[0001]={0001, 0010, 0100, 1000},[0011]={0011, 0110, 1100, 1001},[0111]={0111, 1110, 1101, 1011},(15)

so that the resulting code is commafree? Yes: One solution in this case is simply to choose the smallest word in each class, namely 0001, 0011, and 0111. (Alert readers will recall that we studied the smallest word in the cyclic class of any aperiodic string in Section 7.2.1.1, where such words were called prime strings and where some of the remarkable properties of prime strings were proved.)

That trick doesn’t work when m = 3, however, when there are (81 – 9)/4 = 18 cyclic classes. Then we cannot include 1112 after we’ve chosen 0001 and 0011. Indeed, a code that contains 0001 and 1112 can’t contain either 0011 or 0111.

We could systematically backtrack through 18 levels, choosing x_1 in [0001] and x_2 in [0011], etc., and rejecting each xl as in Algorithm B whenever we discover that {x_1,x_2,...,xl} isn’t commafree. For example, if x_1 = 0010 and we try x_2 = 1001, this approach would backtrack because x_1 occurs inside x_2x_1.

But a naïve strategy of that kind, which recognizes failure only after a bad choice has been made, can be vastly improved. If we had been clever enough, we could have looked a little bit ahead, and never even considered the choice x_2 = 1001 in the first place. Indeed, after choosing x_1 = 0010, we can automatically exclude all further words of the form *001, such as 2001 when m ≥ 3 and 3001 when m ≥ 4.

Even better pruning occurs if, for example, we’ve chosen x_1 = 0001 and x_2 = 0011. Then we can immediately rule out all words of the forms 1*** or ***0, because x_11*** includes x_2 and ***0x_2 includes x_1. Already we could then deduce, in the case m ≥ 3, that classes [0002], [0021], [0111], [0211], and [1112] must be represented by 0002, 0021, 0111, 0211, and 2111, respectively; each of the other three possibilities in those classes has been wiped out!

Thus we see the desirability of a lookahead mechanism.

Dynamic ordering of choices

Furthermore, we can see from this example that it’s not always good to choose x_1, then x_2, then x_3, and so on when trying to satisfy a general property Pn(x_1,x_2,...,xn) in the setting of Algorithm B. Maybe the search tree will be much smaller if we first choose x_5, say, and then turn next to some other xj, depending on the particular value of x_5 that was selected. Some orderings might have much better cutoff properties than others, and every branch of the tree is free to choose its variables in any desired order.

Indeed, our commafree coding problem for ternary 4-tuples doesn’t dictate any particular ordering of the 18 classes that would be likely to keep the search tree small. Therefore, instead of calling those choices x_1, x_2, ..., x_18, it’s better to identify them by the various class names, namely x_0001, x_0002, x_0011, x_0012, x_0021, x_0022, x_0102, x_0111, x_0112, x_0121, x_0122, x_0211, x_0212, x_0221, x_0222, x_1112, x_1122, x_1222. (Algorithm 7.2.1.1F is a good way to generate those names.) At every node of the search tree we then can choose a convenient variable on which to branch, based on previous choices. After beginning with x_0001 ← 0001 at level 1 we might decide to try x_0011 ← 0011 at level 2; and then, as we’ve seen, the choices x_0002 ← 0002, x_0021 ← 0021, x_0111 ← 0111, x_0211 ← 0211, and x_1112 ← 2111 are forced, so we should make them at levels 3 through 7.

Furthermore, after those forced moves are made, it turns out that they don’t force any others. But only two choices for x_0012 will remain, while x_0122 will have three. Therefore it will probably be wiser to branch on x_0012 rather than on x_0122 at level 8. (Incidentally, it also turns out that there is no commafree code of length (m^4 – m^2)/4 with x_0001 = 0001 and x_0011 = 0011, except when m = 2.)

It’s easy to adapt Algorithms B and W to allow dynamic ordering. Every node of the search tree can be given a “frame” in which we record the variable being set and the choice that was made. This choice of variable and value can be called a “move” made by the backtrack procedure.

Dynamic ordering can be helpful also after backtracking has taken place. If we continue the example above, where x_0001 = 0001 and we’ve explored all cases in which x_0011 = 0011, we aren’t obliged to continue by trying another value for x_0011. We do want to remember that 0011 should no longer be considered legal, until x_0001 changes; but we could decide to explore next a case such as x_0002 ← 2000 at level 2. In fact, x_0002 = 2000 is quickly seen to be impossible in the presence of 0001 (see exercise 39). An even more efficient choice at level 2 turns out to be x_0012 ← 0012, because that branch immediately forces x_0002 ← 0002, x_0022 ← 0022, x_0122 ← 0122, x_0222 ← 0222, x_1222 ← 1222, and x_0011 ← 1001.

Sequential allocation redux

The choice of a variable and value on which to branch is a delicate tradeoff. We don’t want to devote more time to planning than we’ll save by having a good plan.

If we’re going to benefit from dynamic ordering, we’ll need efficient data structures that will lead to good decisions without much deliberation. On the other hand, elaborate data structures need to be updated whenever we branch to a new level, and they need to be downdated whenever we return from that level. Algorithm L illustrates an efficient mechanism based on linked lists; but sequentially allocated lists are often even more appealing, because they are cache-friendly and they involve fewer accesses to memory.

Assume then that we wish to represent a set of items as an unordered sequential list. The list begins in a cell of memory pointed to by HEAD, and TAIL points just beyond the end of the list. For example,

is one way to represent the set {1, 3, 4, 9}. The number of items currently in the set is TAIL – HEAD; thus TAIL = HEAD if and only if the list is empty. If we wish to insert a new item x, knowing that x isn’t already present, we simply set

MEM[TAIL]← x,        TAIL ← TAIL +1.(17)

Conversely, if HEAD ≤ P < TAIL, we can easily delete MEM[P]:

TAIL ← TAIL −1;    if P ≠ TAIL;  set  MEM[P] ← MEM[TAIL].(18)

(We’ve tacitly assumed in (17) that MEM[TAIL] is available for use whenever a new item is inserted. Otherwise we would have had to test for memory overflow.)

We can’t delete an item from a list without knowing its MEM location. Thus we will often want to maintain an “inverse list,” assuming that all items x lie in the range 0 ≤ x < M. For example, (16) becomes the following, if M = 10:

(Shaded cells have undefined contents.) With this setup, insertion (17) becomes

MEM[TAIL] ← x,    MEM[IHEAD + x] ← TAIL,   TAIL ← TAIL + 1,(20)

and TAIL will never exceed HEAD + M. Similarly, deletion of x becomes

P← MEM[IHEAD + x][TAIL ←TAIL −1;if P≠TAIL, set  y⁢← MEM[TAIL], MEM[P] ← y, MEM[IHEAD +y] ← P.(21)

For example, after deleting ‘9’ from (19) we would obtain this:

In more elaborate situations we also want to test whether or not a given item x is present. If so, we can keep more information in the inverse list. A particularly useful variation arises when the list that begins at IHEAD contains a complete permutation of the values {HEAD, HEAD + 1, ..., HEAD + M – 1}, and the memory cells beginning at HEAD contain the inverse permutation — although only the first TAIL – HEAD elements of that list are considered to be “active.”

For example, in our commafree code problem with m = 3, we can begin by putting items representing the M = 18 cycle classes [0001], [0002], ..., [1222] into memory cells HEAD through HEAD + 17. Initially they’re all active, with TAIL = HEAD + 18 and MEM[IHEAD + c] = HEAD + c for 0 ≤ c < 18. Then whenever we decide to choose a codeword for class c, we delete c from the active list by using a souped-up version of (21) that maintains full permutations:

P←MEM[IHEAD + c],   TAIL ← TAIL−1;         if P≠TAIL, set, y←MEM[TAIL],    MEM[TAIL] ←c,     MEM[P]← y, ⁢                                                         MEM[IHEAD +c]←TAIL,    MEM[IHEAD +y] ← P.(23)

Later on, after backtracking to a state where we once again want c to be considered active, we simply set TAIL ← TAIL + 1, because c will already be in place! (This data-structuring technique has been called a sparse-set representation; see P. Briggs and L. Torczon, ACM Letters Prog. Lang. and Syst. 2 (1993), 59–69.)

Lists for the commafree problem

The task of finding all four-letter comma-free codes of maximum length is not difficult when m = 3 and only 18 cycle classes are involved. But it already becomes challenging when m = 4, because we must then deal with (4^4 – 4^2)/4 = 60 classes. Therefore we’ll want to give it some careful thought as we try to set it up for backtracking.

The example scenarios for m = 3 considered above suggest that we’ll repeatedly want to know the answers to questions such as, “How many words of the form 02** are still available for selection as codewords?” Redundant data structures, oriented to queries of that kind, appear to be needed. Fortunately, we shall see that there’s a nice way to provide them, using sequential lists as in (19)–(23).

In Algorithm C below, each of the m^4 four-letter words is given one of three possible states during the search for commafree codes. A word is green if it’s part of the current set of tentative codewords. It is red if it’s not currently a candidate for such status, either because it is incompatible with the existing green words or because the algorithm has already examined all scenarios in which it is green in their presence. Every other word is blue, and sort of in limbo; the algorithm might or might not decide to make it red or green. All words are initially blue — except for the m^2 periodic words, which are permanently red.

We’ll use the Greek letter α to stand for the integer value of a four-letter word x in radix m. For example, if m = 3 and if x is the word 0102, then α = (0102)3 = 11. The current state of word x is kept in MEM[α], using one of the arbitrary internal codes 2 (GREEN), 0 (RED), or 1 (BLUE).

The most important feature of the algorithm is that every blue word x = x_1x_2x_3x_4 is potentially present in seven different lists, called P1(x), P2(x), P3(x), S1(x), S2(x), S3(x), and CL(x), where

- P1(x), P2(x), P3(x) are the blue words matching x_1***, x_1x_2**, x_1x_2x_3*;

- S1(x), S2(x), S3(x) are the blue words matching ***x_4, **x_3x_4, *x_2x_3x_4;

- CL(x) hosts the blue words in {x_1x_2x_3x_4,x_2x_3x_4x_1,x_3x_4x_1x_2,x_4x_1x_2x_3}.

These seven lists begin respectively in MEM locations P1OFF + p_1(α), P2OFF + p_2(α), P3OFF + p_3(α), S1OFF + s_1(α), S2OFF + s_2(α), S3OFF + s_3(α), and CLOFF + 4cl(α); here (P1OFF, P2OFF, P3OFF, S1OFF, S2OFF, S3OFF, CLOFF) are respectively (2m^4, 5m^4, 8m^4, 11m^4, 14m^4, 17m^4, 20m^4). We define p_1((x_1x_2x_3x_4)m)= (x_1000)m, p_2((x_1x_2x_3x_4)m) = (x_1x_200)m, p_3((x_1x_2x_3x_4)m) = (x_1x_2x_30)m; and s_1((x_1x_2x_3x_4)m)=(x_4000)m, s_2((x_1x_2x_3x_4)m)=(x_3x_400)m, s_3((x_1x_2x_3x_4)m)= (x_2x_3x_40)m; and finally cl((x_1x_2x_3x_4)m) is an internal number, between 0 and (m^4–m^2)/4–1, assigned to each class. The seven MEM locations where x appears in these seven lists are respectively kept in inverse lists that begin in MEM locations P1OFF – m^4 + α, P2OFF – m^4 + α, ..., CLOFF – m^4 + α. And the TAIL pointers, which indicate the current list sizes as in (19)–(23), are respectively kept in MEM locations P1OFF + m^4 + p_1(α), P2OFF + m^4 + p_2(α), ..., S3OFF + m^4 + s_3(α), CLOFF + m^4 +4cl(α). (Whew; got that?)

This vast apparatus, which occupies 22m^4 cells of MEM, is illustrated in Table 1, at the beginning of the computation for the case m = 2. Fortunately it’s not really as complicated as it may seem at first. Nor is it especially vast: After all, 22m^4 is only 13,750 when m = 5.

Table 1
LISTS USED BY ALGORITHM C (m = 2), ENTERING LEVEL 1

0

1

2

3

4

5

6

7

8

9

a

b

c

d

e

f

0

RED

BLUE

BLUE

BLUE

RED

RED

BLUE

BLUE

RED

BLUE

RED

BLUE

BLUE

BLUE

BLUE

RED

10

20

21

22

23

24

29

2c

28

2b

2a

20

0001

0010

0011

0110

0111

1100

1001

1110

1101

1011

P1

30

25

2d

40

50

51

52

54

55

58

59

5c

5e

5d

50

0001

0010

0011

0110

0111

1001

1011

1100

1110

1101

P2

60

53

56

5a

5f

70

80

82

83

86

87

88

8a

8c

8d

8e

80

0001

0010

0011

0110

0111

1001

1011

1100

1101

1110

P3

90

81

84

84

88

89

8b

8e

8f

a0

b8

b0

b9

b1

bb

ba

bd

b2

bc

b3

b0

0010

0110

1100

1110

0001

0011

1001

0111

1101

1011

S1

c0

b4

be

d0

e4

e8

ec

e9

ed

e5

ee

e0

e6

ea

e0

1100

0001

1001

1101

0010

0110

1110

0011

0111

1011

S2

f0

e1

e7

eb

ef

100

112

114

116

11c

11e

113

117

118

11a

11d

110

0001

1001

0010

0011

1011

1100

1101

0110

1110

0111

S3

120

110

114

115

118

119

11b

11e

11f

130

140

141

144

145

148

147

14b

146

14a

149

140

0001

0010

0011

0110

1100

1001

0111

1110

1101

1011

CL

150

142

148

14c

This table shows MEM locations 0000 through 150f, using hexadecimal notation. (For example, MEM[40d]=5e; see exercise 41.) Blank entries are unused by the algorithm.

(A close inspection of Table 1 reveals incidentally that the words 0100 and 1000 have been colored red, not blue. That’s because we can assume without loss of generality that class [0001] is represented either by 0001 or by 0010. The other two cases are covered by left-right reflection of all codewords.)

Algorithm C finds these lists invaluable when it is deciding where next to branch. But it has no further use for a list in which one of the items has become green. Therefore it declares such lists “closed”; and it saves most of the work of list maintenance by updating only the lists that remain open. A closed list is represented internally by setting its TAIL pointer to HEAD – 1.

For example, Table 2 shows how the lists in MEM will have changed just after x = 0010 has been chosen to be a tentative codeword. The elements {0001, 0010, 0011, 0110, 0111} of P1(x) are effectively hidden, because the tail pointer MEM[30] = 1f = 20–1 marks that list as closed. (Those list elements actually do still appear in MEM locations 200 through 204, just as they did in Table 1. But there’s no need to look at that list while any word of the form 0*** is green.)

A general mechanism for doing and undoing

We’re almost ready to finalize the details of Algorithm C and to get on with the search for commafree codes, but a big problem still remains: The state of computation at every level of the search involves all of the marvelous lists that we’ve just specified, and those lists aren’t tiny. They occupy more than 5000 cells of MEM when m = 4, and they can change substantially from level to level.

We could make a new copy of the entire state, whenever we advance to a new node of the search tree. But that’s a bad idea, because we don’t want to perform thousands of memory accesses per node. A much better strategy would be to stick with a single instance of MEM, and to update and downdate the lists as the search progresses, if we could only think of a simple way to do that.

And we’re in luck: There is such a way, first formulated by R. W. Floyd in his classic paper “Nondeterministic algorithms” [JACM 14 (1967), 636–644]. Floyd’s original idea, which required a special compiler to generate forward and backward versions of every program step, can in fact be greatly simplified when all of the changes in state are confined to a single MEM array. All we need to do is to replace every assignment operation of the form ‘MEM[a] ← v’ by the slightly more cumbersome operation
store(a,v):          Set   UNDO[u] ← (a,MEM[a]),    MEM[a]← v,and  u← u+1.(24)

Here UNDO is a sequential stack that holds (address, value) pairs; in our application we could say ‘UNDO[u] ← (a ≪ 16) + MEM[a]’, because the cell addresses and values never exceed 16 bits. Of course we’ll also need to check that the stack pointer u doesn’t get too large, if the number of assignments has no a priori limit.

Later on, when we want to undo all changes to MEM that were made after the time when u had reached a particular value u_0, we simply do this:

unstore(u0):     While  u⁢> u0,   set  u⁢ ← u⁢−1,                                                               (a,v)← UNDO[u],     and MEM[a]←v.(25)

In our application the unstacking operation ‘(a, v) ← UNDO[u]’ here could be implemented by saying ‘a ← UNDO[u] ≫ 16, v ← UNDO[u] & # ffff’.

A useful refinement of this reversible-memory technique is often advantageous, based on the idea of “stamping” that is part of the folklore of programming. It puts only one item on the UNDO stack when the same memory address is updated more than once in the same round.

store⁢ (a,v) :If STAMP [a]≠σ,setSTAMP[a]←σ,           UNDO[u]←(a, MEM[a]), and   u⁢ ← u+1.Then set MEM[a] ←v.(26)

Here STAMP is an array with one entry for each address in MEM. It’s initially all zero, and σ is initially 1. Whenever we come to a fallback point, where the current stack pointer will be remembered as the value u_0 for some future undoing, we “bump” the current stamp by setting σ ← σ + 1. Then (26) will continue to do the right thing. (In programs that run for a long time, we must be careful when integer overflow causes σ to be bumped to zero; see exercise 43.)

Notice that the combination of (24) and (25) will perform five memory accesses for each assignment and its undoing. The combination of (26) and (25) will cost seven mems for the first assignment to MEM[a], but only two mems for every subsequent assignment to the same address. So (26) wins, if multiple assignments exceed one-time-only assignments.

Backtracking through commafree codes

OK, we’re now equipped with enough basic knowhow to write a pretty good backtrack program for the problem of generating all commafree four-letter codes.

Algorithm C below incorporates one more key idea, which is a lookahead mechanism that is specific to commafree backtracking; we’ll call it the “poison list.” Every item on the poison list is a pair, consisting of a suffix and a prefix that the commafree rule forbids from occurring together. Every green word x_1x_2x_3x_4 — that is, every word that will be a final codeword in the current branch of our backtrack search — contributes three items to the poison list, namely

(*x1x2x3, x4),     (**x1x2,x3x4**),         and   (***x1, x2x3x4*).(27)

If there’s a green word on both sides of a poison list entry, we’re dead: The commafree condition fails, and we mustn’t proceed. If there’s a green word on one side but not the other, we can kill off all blue words on the other side by making them red. And if either side of a poison list entry corresponds to an empty list, we can remove this entry from the poison list because it will never affect the outcome. (Blue words become red or green, but red words stay red.)

For example, consider the transition from Table 1 to Table 2. When word 0010 becomes green, the poison list receives its first three items:

Table 2
LISTS USED BY ALGORITHM C (m = 2), ENTERING LEVEL 2

0

1

2

3

4

5

6

7

8

9

a

b

c

d

e

f

0

RED

RED

GREEN

BLUE

RED

RED

BLUE

BLUE

RED

RED

RED

BLUE

BLUE

BLUE

BLUE

RED

10

29

28

2b

2a

20

1100

1011

1110

1101

P1

30

1f

2c

40

54

55

58

5c

5e

5d

50

0110

0111

1011

1100

1110

1101

P2

60

4f

56

59

5f

70

86

87

8a

8c

8d

8e

80

0110

0111

1011

1100

1101

1110

P3

90

80

81

84

88

88

8b

8e

8f

a0

b9

bb

b8

ba

b0

1011

0011

1101

0111

S1

c0

af

bc

d0

ec

ed

ee

e0

e4

e0

1100

1101

0011

0111

1011

S2

f0

e1

e5

e7

ef

100

116

11c

11e

117

118

11a

11d

110

0011

1011

1100

1101

0110

1110

0111

S3

120

110

112

113

118

119

11b

11e

11f

130

144

145

148

14b

146

14a

149

140

0011

0110

1100

0111

1110

1101

1011

CL

150

13f

147

14c

The word 0010 has become green, thus closing its seven lists and making 0001 red. The logic of Algorithm C has also made 1001 red. Hence 0001 and 1001 have been deleted from the open lists in which they formerly appeared (see exercise 42).

(*001, 0***),   (**00, 10**),   (***0, 010*).

The first of these kills off the *001 list, because 0*** contains the green word 0010. That makes 1001 red. The last of these, similarly, kills off the 010* list; but that list is empty when m = 2. The poison list now reduces to a single item, (**00, 10**), which remains poisonous because list **00 contains the blue word 1100 and 10** contains the blue word 1011.

We’ll maintain the poison list at the end of MEM, following the CL lists. It obviously will contain at most 3(m^4 – m^2)/4 entries, and in fact it usually turns out to be quite small. No inverse list is required; so we shall adopt the simple method of (17) and (18), but with two cells per entry so that TAIL will change by ±2 instead of by ±1. The value of TAIL will be stored in MEM at key times so that temporary changes to it can be undone.

The case m = 4, in which each codeword consists of four quaternary digits {0, 1, 2, 3}, is particularly interesting, because an early backtrack program by Lee Laxdal found that no such commafree code can make use of all 60 of the cycle classes [0001], [0002], ..., [2333]. [See B. H. Jiggs, Canadian Journal of Math. 15 (1963), 178–187.] Laxdal’s program also reportedly showed that at least three of those classes must be omitted; and it found several valid 57-word sets. Further details were never published, because the proof that 58 codewords are impossible depended on what Jiggs called a “quite time-consuming” computation.

Because size 60 is impossible, our algorithm cannot simply assume that a move such as 1001 is forced when the other words 0011, 0110, 1100 of its class have been ruled out. We must also consider the possibility that class [0011] is entirely absent from the code. Such considerations add an interesting further twist to the problem, and Algorithm C describes one way to cope with it.

Algorithm C (Four-letter commafree codes). Given an alphabet size m ≤ 7 and a goal g in the range L – m(m – 1) ≤ g ≤ L, where L = (m^4 – m^2)/4, this algorithm finds all sets of g four-letter words that are commafree and include either 0001 or 0010. It uses an array MEM of M = ⌊23.5m^4⌋ 16-bit numbers, as well as several more auxiliary arrays: ALF of size 16^3m; STAMP of size M; X, C, S, and U of size L +1; FREE and IFREE of size L; and a sufficiently large array called UNDO whose maximum size is difficult to guess.

C1. [Initialize.] Set ALF[(abcd)16] ← (abcd)m for 0 ≤ a, b, c, d < m. Set STAMP[k] ← 0 for 0 ≤ k < M and σ ← 0. Put the initial prefix, suffix, and class lists into MEM, as in Table 1. Also create an empty poison list by setting MEM[PP] ← POISON, where POISON = 22m^4 and PP = POISON – 1. Set FREE[k] ← IFREE[k] ← k for 0 ≤ k < L. Then set l ← 1, x ←#0001, c ← 0, s ← L – g, f ← L, u ← 0, and go to step C3. (Variable l is the level, x is a trial word, c is its class, s is the “slack,” f is the number of free classes, and u is the size of the UNDO stack.)

C2. [Enter level l.] If l > L, visit the solution x_1 ... xL and go to C6. Otherwise choose a candidate word x and class c as described in exercise 44.

C3. [Try the candidate.] Set U[l] ← u and σ ← σ +1. If x < 0, go to C6 if s = 0 or l = 1, otherwise set s ← s – 1. If x ≥ 0, update the data structures to make x green, as described in exercise 45, escaping to C5 if trouble arises.

C4. [Make the move.] Set X[l] ← x, C[l] ← c, S[l] ← s, p ← IFREE[c], f ← f –1. If p ≠ f, set y ← FREE[f], FREE[p] ← y, IFREE[y] ← p, FREE[f] ← c, IFREE[c] ← f. (This is (23).) Then set l ← l + 1 and go to C2.

C5. [Try again.] While u > U[l], set u ← u – 1 and MEM[UNDO[u] ≫ 16] ← UNDO[u] & # ffff. (Those operations restore the previous state, as in (25).) Then σ ← σ + 1 and redden x (see exercise 45). Go to C2.

C6. [Backtrack.] Set l ← l – 1, and terminate if l = 0. Otherwise set x ← X[l], c ← C[l], f ← f +1. If x < 0, repeat this step (class c was omitted from the code). Otherwise set s ← S[l] and go back to C5.

Exercises 44 and 45 provide the instructive details that flesh out this skeleton.

Algorithm C needs just 13, 177, and 2380 megamems to prove that no solutions exist for m = 4 when g is 60, 59, and 58. It needs about 22800 megamems to find the 1152 solutions for g = 57; see exercise 47. There are roughly (14, 240, 3700, 38000) thousand nodes in the respective search trees, with most of the activity taking place on levels 30 ± 10. The height of the UNDO stack never exceeds 2804, and the poison list never contains more than 12 entries at a time.

Running time estimates

Backtrack programs are full of surprises. Sometimes they produce instant answers to a supposedly difficult problem. But sometimes they spin their wheels endlessly, trying to traverse an astronomically large search tree. And sometimes they deliver results just about as fast as we might expect.

Fortunately, we needn’t sit in the dark. There’s a simple Monte Carlo algorithm by which we can often tell in advance whether or not a given backtrack strategy will be feasible. This method, based on random sampling, can actually be worked out by hand before writing a program, in order to help decide whether to invest further time while following a particular approach. In fact, the very act of carrying out this pleasant pencil-and-paper method often suggests useful cutoff strategies and/or data structures that will be valuable later when a program is being written. For example, the author developed Algorithm C above after first doing some armchair experiments with random choices of potential commafree codewords; these dry runs revealed that a family of lists such as those in Tables 1 and 2 would be quite helpful when making further choices.

To illustrate the method, let’s consider the n queens problem again, as represented in Algorithm B* above. When n = 8, we can obtain a decent “ballpark estimate” of the size of Fig. 68 by examining only a few random paths in that search tree. We start by writing down the number D_1 ← 8, because there are eight ways to place the queen in row 1. (In other words, the root node of the search tree has degree 8.) Then we use a source of random numbers — say the binary digits of π mod1 = (.001001000011 ... )2 — to select one of those placements. Eight choices are possible, so we look at three of those bits; we shall set X_1 ← 2, because 001 is the second of the eight possibilities (000, 001, ..., 111).

Given X_1 = 2, the queen in row 2 can’t go into columns 1, 2, or 3. Hence five possibilities remain for X_2, and we write down D_2 ← 5. The next three bits of π lead us to set X_2 ← 5, since 5 is the second of the available columns (4, 5, 6, 7, 8) and 001 is the second value of (000, 001, ..., 100). Incidentally, if π had continued with 101 or 110 or 111 instead of 001, we would have used the “rejection method” of Section 3.4.1 and moved to the next three bits; see exercise 49.

Continuing in this way leads to D_3 ← 4, X_3 ← 1; then D_4 ← 3, X_4 ← 4. (Here we used the two bits 00 to select X_3, and the next two bits 00 to select X_4.) The remaining branches are forced: D_5 ← 1, X_5 ← 7; D_6 ← 1, X_6 ← 3; D_7 ← 1, X_7 ← 6; and we’re stuck when we reach level 8 and find D_8 ← 0.

These sequential random choices are depicted in Fig. 69(a), where we’ve used them to place each queen successively into an unshaded cell. Parts (b), (c), and (d) of Fig. 69 correspond in the same way to choices based on the binary digits of e mod 1, ϕ mod 1, and γ mod 1. Exactly 10 bits of π, 20 bits of e, 13 bits of ϕ, and 13 bits of γ were used to generate these examples.

Fig. 69. Four random attempts to solve the 8 queens problem. Such experiments help to estimate the size of the backtrack tree in Fig. 68. The branching degrees are shown at the right of each diagram, while the random bits used for sampling appear below. Cells have been shaded in gray if they are attacked by one or more queens in earlier rows.

In this discussion the notation Dk stands for a branching degree, not for a domain of values. We’ve used uppercase letters for the numbers D_1, X_1, D_2, etc., because those quantities are random variables. Once we’ve reached Dl = 0 at some level, we’re ready to estimate the overall cost, by implicitly assuming that the path we’ve taken is representative of all root-to-leaf paths in the tree.

The cost of a backtrack program can be assessed by summing the individual amounts of time spent at each node of the search tree. Notice that every node on level l of that tree can be labeled uniquely by a sequence x_1 ... xl–1, which defines the path from the root to that node. Thus our goal is to estimate the sum of all c(x_1 ... xl–1), where c(x_1 ... xl–1) is the cost associated with node x_1 ... xl–1.

For example, the four queens problem is represented by the search tree (4), and its cost is the sum of 17 individual costs
c() + c(1) + c(13)  + c(14)+ c(142) + c(2) + c(24) + .⁢ .⁢ ⁢.⁢ +  c(413) + c(42).(28)

If C(x_1 ... xl) denotes the total cost of the subtree rooted at x_1 ... xl, then

C(x1⁢ .⁢.⁢.⁢ xl) =c(x1⁢ .⁢.⁢.⁢ xl) + C(x1⁢ .⁢.⁢.⁢ xlxl+1(1)) + .⁢.⁢.⁢  + C(x1⁢ .⁢.⁢.⁢ xlxl+1(d))(29)

when the choices for xl + 1 at node x_1 ... xl are {xl+1(1), .⁢.⁢.⁢ ,xl+1(d)}. For instance in (4) we have C(1) = c(1) + C(13) + C(14); C(13) = c(13); and C() = c() + C(1) + C(2) + C(3) + C(4) is the overall cost (28).

In these terms a Monte Carlo estimate for C() is extremely easy to compute:

Theorem E. Given D_1, X_1, D_2, X_2, ... as above, the cost of backtracking is

C() = E(c() + D1(c(X1) +D2(c(X1X2) + D3(c(X1X2X3) + .⁢ .⁢ .)))).(30)

Proof. Node x_1 ... xl, with branch degrees d_1, ..., dl above it, is reached with probability 1 / d_1 ... dl; so it contributes d_1 ... dlc(x_1 ... xl) / d_1 ... dl = c(x_1 ... xl) to the expected value in this formula.

For example, the tree (4) has six root-to-leaf paths, and they occur with respective probabilities 1/8, 1/8, 1/4, 1/4, 1/8, 1/8. The first one contributes 1/8 times c() + 4(c (1) + 2(c(13))), namely c() / 8 + c(1) / 2 + c(13), to the expected value. The second contributes c() / 8 + c(1) / 2 + c(14) + c(142); and so on.

A special case of Theorem E, with all c(x_1 ... xl) = 1, tells us how to estimate the total size of the tree, which is often a crucial quantity:

Corollary E. The number of nodes in the search tree, given D_1, D_2, ... , is

E(1+D1⁢ +D1 D2⁢ + .⁢ .⁢ .) = E(1+D1(1+D2(1+D3(1+.⁢ .⁢ .⁢ )))).(31)

For example, Fig. 69 gives us four estimates for the size of the tree in Fig. 68, using the numbers Dj at the right of each 8 × 8 diagram. The estimate from Fig. 69(a) is 1 + 8 (1 + 5 (1 + 4(1 + 3(1 + 1(1 + 1(1 + 1)))))) = 2129; and the other three are respectively 2689, 1489, 2609. None of them is extremely far from the true number, 2057, although we can’t expect to be so lucky all the time.

The detailed study in exercise 53 shows that the estimate (31) in the case of 8 queens turns out to be quite well behaved:

(min 489,    ave 2057,     max 7409,       dev 1146640⁢ ≈ 1071).(32)

The analogous problem for 16 queens has a much less homogeneous search tree:

(min 2597105,   ave 1141190303,  max 131048318769,  dev⁢ ≈ 1234000000).(33)

Still, this standard deviation is roughly the same as the mean, so we’ll usually guess the correct order of magnitude. (For example, ten independent experiments predicted .632, .866, .237, 1.027, 4.006, .982, .143, .140, 3.402, and .510 billion nodes, respectively. The mean of these is 1.195.) A thousand trials with n = 64 suggest that the problem of 64 queens will have about 3 × 10^65 nodes in its tree.

Let’s formulate this estimation procedure precisely, so that it can be performed conveniently by machine as well as by hand:

Algorithm E (Estimated cost of backtrack). Given domains Dk and properties Pl as in Algorithm B, together with node costs c(x_1 ... xl) as above, this algorithm computes the quantity S whose expected value is the total cost C() in (30). It uses an auxiliary array y_0y_1 ... whose size should be ≥ max(|D_1|, ..., |Dn|).

E1. [Initialize.] Set l ← D ← 1, S ← 0, and initialize any data structures needed.

E2. [Enter level l.] (At this point Pl−1(X_1, ..., Xl−1) holds.) Set S ← S + D · c(X_1 ... Xl−1). If l > n, terminate the algorithm. Otherwise set d ← 0 and set x ← min Dl, the smallest element of Dl.

E3. [Test x.] If Pl(X_1,..., Xl−1, x) holds, set yd ← x and d ← d +1.

E4. [Try again.] If x ≠ max Dl, set x to the next larger element of Dl and return to step E3.

E5. [Choose and try.] If d = 0, terminate. Otherwise set D ← D·d and Xl ← yI, where I is a uniformly random integer in {0, ..., d − 1}. Update the data structures to facilitate testing Pl+1, set l ← l + 1, and go back to E2.

Although Algorithm E looks rather like Algorithm B, it never backtracks.

Of course we can’t expect this algorithm to give decent estimates in cases where the backtrack tree is wildly erratic. The expected value of S, namely E S, is indeed the true cost; but the probable values of S might be quite different.

An extreme example of bad behavior occurs if property Pl is the simple condition ‘x_1 > … > xl’, and all domains are {1, ..., n}. Then there’s only one solution, x_1 ... xn = n ... 1; and backtracking is a particularly stupid way to find it!

The search tree for this somewhat ridiculous problem is, nevertheless, quite interesting. It is none other than the binomial tree Tn of Eq. 7.2.1.3–(21), which has (nl) nodes on level l + 1 and 2n nodes in total. If we set all costs to 1, expected value of S is therefore 2n = en ln 2. But exercise 52 proves that S will almost always be much smaller, less than e(ln n)2 ln ln n. Furthermore the average value of l when Algorithm E terminates with respect to Tn is only Hn + 1. When n = 100, for example, the probability that l ≥ 20 on termination is only 0.0000000027, while the vast majority of the nodes are near level 51.

Many refinements of Algorithm E are possible. For example, exercise 54 shows that the choices in step E5 need not be uniform. We shall discuss improved estimation techniques in Section 7.2.2.9, after having seen numerous examples of backtracking in practice.

*Estimating the number of solutions

Sometimes we know that a problem has more solutions than we could ever hope to generate, yet we still want to know roughly how many there are. Algorithm E will tell us the approximate number, in cases where the backtrack process never reaches a dead end — that is, if it never terminates with d = 0 in step E5. There may be another criterion for successful termination in step E2 even though l might still be ≤ n. The expected final value of D is exactly the total number of solutions, because every solution X_1 ... Xl constructed by the algorithm is obtained with probability 1/D.

For example, suppose we want to know the number of different paths by which a king can go from one corner of a chessboard to the opposite corner, without revisiting any square. One such path, chosen at random using the bits of π for guidance as we did in Fig. 69(a), is shown here. Starting in the upper left corner, we have 3 choices for the first move. Then, after moving to the right, there are 4 choices for the second move. And so on. We never make a move that would disconnect us from the goal; in particular, two of the moves are actually forced. (Exercise 58 explains one way to avoid fatal mistakes.)

The probability of obtaining this particular path is exactly 13141616121617⁢.⁢ .⁢ .⁢ 12⁢=1/D, where D = 3 × 4 × 6 × 6 × 2 × 6 × 7 × … × 2 = 1^2 · 2^4 · 3^4 · 4^10 · 5^9 · 6^6 · 7^1 ≈ 8.7 × 10^20. Thus we can reasonably guess, at least tentatively, that there are 10^21 such paths, more or less.

Of course that guess, based on a single random sample, rests on very shaky ground. But we know that the average value MN = (D(1) + … + D(N)) / N of N guesses, in N independent experiments, will almost surely approach the correct number.

How large should N be, before we can have any confidence in the results? The actual values of D obtained from random king paths tend to vary all over the map. Figure 70 plots typical results, as N varies from 1 to 10000. For each value of N we can follow the advice of statistics textbooks and calculate the sample variance VN = SN /(N − 1) as in Eq. 4.2.2–(16); then MN⁢ ± VN/N is the textbook estimate. The top diagram in Fig. 70 shows these “error bars” in gray, surrounding black dots for MN. This sequence MN does appear to settle down after N reaches 3000 or so, and to approach a value near 5 × 10^25. That’s much higher than our first guess, but it has lots of evidence to back it up.

On the other hand, the bottom chart in Fig. 70 shows the distribution of the logarithms of the 10000 values of D that were used to make the top chart. Almost half of those values were totally negligible — less than 10^20. About 75% of them were less than 10^24. But some of them* exceeded 10^28. Can we really rely on a result that’s based on such chaotic behavior? Is it really right to throw away most of our data and to trust almost entirely on observations that were obtained from comparatively few rare events?

* Four of the actual values that led to Fig. 70 were larger than 10^28; the largest, ≈ 2.1 × 10^28, came from a path of length 57. The smallest estimate, 19361664, came from a path of length 10.

Fig. 70. Estimates of the number of king paths, based on up to 10000 random trials. The middle graph shows the corresponding quality measures of Eq. (34). The lower graph shows the logarithms of the individual estimates D(k), after they’ve been sorted.

Yes, we’re okay! Some of the justification appears in exercise MPR–124, which is based on theoretical work by S. Chatterjee and P. Diaconis. In the paper cited with that exercise, they defend a simple measure of quality,

X^N⁢ =max(D(1),  .⁢ .⁢ .⁢ ,D(N)/(NMN) =  max(D(1), .⁢ .⁢ ⁢.⁢ , D(N))D(1)⁢ + .⁢⁢.⁢ ⁢.⁢ + D(N),(34)

arguing that a reasonable policy in most experiments such as these is to stop sampling when X^N gets small. (Values of this statistic X^N have been plotted in the middle of Fig. 70.)

Furthermore we can estimate other properties of the solutions to a backtrack problem, instead of merely counting those solutions. For example, the expected value of lD on termination of the random king’s path algorithm is the total length of such paths. The data underlying Fig. 70 suggests that this total is (2.66 ± .14) × 10^27; hence the average path length appears to be about 53. The samples also indicate that about 34% of the paths pass through the center; about 46% touch the upper right corner; about 22% touch both corners; and about 7% pass through the center and both corners.

For this particular problem we don’t actually need to rely on estimates, because the ZDD technology of Section 7.1.4 allows us to compute the true values. (See exercise 59.) The total number of simple corner-to-corner king paths on a chessboard is exactly 50,819,542,770,311,581,606,906,543; this value lies almost within the error bars of Fig. 70 for all N ≥ 250, except for a brief interval near N = 1400. And the total length of all these paths turns out to be exactly 2,700,911,171,651,251,701,712,099,831, which is a little higher than our estimate. The true average length is therefore ≈ 53.15. The true probabilities of hitting the center, a given corner, both corners, and all three of those spots are respectively about 38.96%, 50.32%, 25.32%, and 9.86%.

The total number of corner-to-corner king paths of the maximum length, 63, is 2,811,002,302,704,446,996,926. This is a number that can not be estimated well by a method such as Algorithm E without additional heuristics.

The analogous problem for corner-to-corner knight paths, of any length, lies a bit beyond ZDD technology because many more ZDD nodes are needed. Using Algorithm E we can estimate that there are about (8.6 ± 1.2) × 10^19 such paths.

Factoring the problem

Imagine an instance of backtracking that is equivalent to solving two independent subproblems. For example, we might be looking for all sequences x = x_1x_2 ... xn that satisfy Pn(x_1, x_2, ..., xn) = F (x_1, x_2, ..., xn), where

F(x1 ,x2 ,.⁢ .⁢ .⁢ ,xn ) = G(x1 , .⁢ .⁢ .⁢ , xk ) Λ⁢ ⁢ H(xk+1,.⁢ .⁢ .⁢ , xn).(35)

Then the size of the backtrack tree is essentially the product of the tree sizes for G and for H, even if we use dynamic ordering. Hence it’s obviously foolish to apply the general setup of (1) and (2). We can do much better by finding all solutions to G first, then finding all solutions to H, thereby reducing the amount of computation to the sum of the tree sizes. Again we’ve divided and conquered, by factoring the compound problem (35) into separate subproblems.

We discussed a less obvious application of problem factorization near the beginning of Chapter 7, in connection with latin squares: Recall that E. T. Parker sped up the solution of 7–(6) by more than a dozen orders of magnitude, when he discovered 7–(7) by essentially factoring 7–(6) into ten subproblems whose solutions could readily be combined.

In general, each solution x to some problem F often implies the existence of solutions x(p) = ϕp(x) to various simpler problems Fp that are “homomorphic images” of F. And if we’re lucky, the solutions to those simpler problems can be combined and “lifted” to a solution of the overall problem. Thus it pays to be on the lookout for such simplifications.

Let’s look at another example. F. A. Schossow invented a tantalizing puzzle [U.S. Patent 646463 (3 April 1900)] that went viral in 1967 when a marketing genius decided to rename it “Instant Insanity.” The problem is to take four cubes such as

where each face has been marked in one of four ways, and to arrange them in a row so that all four markings appear on the top, bottom, front, and back sides. The placement in (36) is incorrect, because there are two ♣s (and no ♠) on top. But we get a solution if we rotate each cube by 90°.

There are 24 ways to place each cube, because any of the six faces can be on top and we can rotate four ways while keeping the top unchanged. So the total number of placements is 24^4 = 331776. But this problem can be factored in an ingenious way, so that all solutions can be found quickly by hand! [See F. de Carteblanche, Eureka 9 (1947), 9–11.] The idea is that any solution to the puzzle gives us two each of , if we look only at the top and bottom or only at the front and back. That’s a much easier problem to solve.

For this purpose a cube can be characterized by its three pairs of markings on opposite faces; in (36) these face-pairs are respectively

Which of the 3^4 = 81 ways to choose one face-pair from each cube will give us ? They can all be discovered in a minute or two, by listing the nine possibilities for cubes (1, 2) and the nine for (3, 4). We get just three,

Notice furthermore that each solution can be “halved” so that one each of appears on both sides, by swapping face-pairs; we can change (38) to

Each of these solutions to the opposite-face subproblem can be regarded as a 2-regular graph, because every vertex of the multigraph whose edges are (say) has exactly two neighbors.

A solution to “Instant Insanity” will give us two such 2-regular factors, one for top-and-bottom and one for front-and-back. Furthermore those two factors will have disjoint edges: We can’t use the same face-pair in both. Therefore problem (36) can be solved only by using the first and third factor in (39).

Conversely, whenever we have two disjoint 2-regular graphs, we can always use them to position the cubes as desired, thus “lifting” the factors to a solution of the full problem.

Exercise 67 illustrates another kind of problem factorization. We can conveniently think of each subproblem as a “relaxation” of constraints.

Historical notes

The origins of backtrack programming are obscure. Equivalent ideas must have occurred to many people, yet there was hardly any reason to write them down until computers existed. We can be reasonably sure that James Bernoulli used such principles in the 17th century, when he successfully solved the “Tot tibi sunt dotes” problem that had eluded so many others (see Section 7.2.1.7), because traces of the method exist in his exhaustive list of solutions.

Backtrack programs typically traverse the tree of possibilities by using what is now called depth-first search, a general graph exploration procedure that Édouard Lucas credited to a telegraph engineer named Trémaux [Récréations Mathématiques 1 (Paris: Gauthier-Villars, 1882), 47.50].

The eight queens problem was first proposed by Max Bezzel [Schachzeitung 3 (1848), 363; 4 (1849), 40] and by Franz Nauck [Illustrirte Zeitung 14, 361 (1 June 1850), 352; 15, 377 (21 September 1850), 182], perhaps independently. C. F. Gauss saw the latter publication, and wrote several letters about it to his friend H. C. Schumacher. Gauss’s letter of 27 September 1850 is especially interesting, because it explained how to find all the solutions by backtracking. which he called ‘Tatonniren’, from a French term meaning “to feel one’s way.” He also listed the lexicographically first solutions of each equivalence class under reflection and rotation: 15863724, 16837425, 24683175, 25713864, 25741863, 26174835, 26831475, 27368514, 27581463, 35281746, 35841726, and 36258174.

Computers arrived a hundred years later, and people began to use them for combinatorial problems. The time was therefore ripe for backtracking to be described as a general technique, and Robert J. Walker rose to the occasion [Proc. Symposia in Applied Math. 10 (1960), 91.94]. His brief note introduced Algorithm W in machine-oriented form, and mentioned that the procedure could readily be extended to find variable-length patterns x_1 ... xn where n is not fixed.

The next milestone was a paper by Solomon W. Golomb and Leonard D. Baumert [JACM 12 (1965), 516–524], who formulated the general problem carefully and presented a variety of examples. In particular, they discussed the search for maximum commafree codes, and noted that backtracking can be used to find successively better and better solutions to combinatorial optimization problems. They introduced certain kinds of lookahead, as well as the important idea of dynamic ordering by branching on variables with the fewest remaining choices.

Backtrack methods allow special cutoffs when applied to integer programming problems [see E. Balas, Operations Research 13 (1965), 517–546]. A. M. Geoffrion simplified and extended that work, calling it “implicit enumeration” because many cases aren’t enumerated explicitly [SIAM Rev. 9 (1967), 178–190].

Other noteworthy early discussions of backtrack programming appear in Mark Wells’s book Elements of Combinatorial Computing (1971), Chapter 4; in a survey by J. R. Bitner and E. M. Reingold, CACM 18 (1975), 651–656; and in the Ph.D. thesis of John Gaschnig [Report CMU-CS-79-124 (Carnegie Mellon University, 1979), Chapter 4]. Gaschnig introduced techniques of “backmarking” and “backjumping” that we shall discuss later.

Monte Carlo estimates of the cost of backtracking were first described briefly by M. Hall, Jr., and D. E. Knuth in Computers and Computing, AMM 72, 2, part 2, Slaught Memorial Papers No. 10 (February 1965), 21–28. Knuth gave a much more detailed exposition a decade later, in Math. Comp. 29 (1975), 121– 136. Such methods can be considered as special cases of so-called “importance sampling”; see J. M. Hammersley and D. C. Handscomb, Monte Carlo Methods (London: Methuen, 1964), 57–59. Studies of random self-avoiding walks such as the king paths discussed above were inaugurated by M. N. Rosenbluth and A. W. Rosenbluth, J. Chemical Physics 23 (1955), 356–359.

Backtrack applications are nicely adaptable to parallel programming, because different parts of the search tree are often completely independent of each other; thus disjoint subtrees can be explored on different machines, with a minimum of interprocess communication. Already in 1964, D. H. Lehmer explained how to subdivide a problem so that two computers of different speeds could work on it simultaneously and finish at the same time. The problem that he considered had a search tree of known shape (see Theorem 7.2.1.3L); but we can do essentially similar load balancing even in much more complicated situations, by using Monte Carlo estimates of the subtree sizes. Although many ideas for parallelizing combinatorial searches have been developed over the years, such techniques are beyond the scope of this book. Readers can find a nice introduction to a fairly general approach in the paper by R. Finkel and U. Manber, ACM Transactions on Programming Languages and Systems 9 (1987), 235–256.

M. Alekhnovich, A. Borodin, J. Buresh-Oppenheim, R. Impagliazzo, A. Magen, and T. Pitassi have defined priority branching trees, a general model of computation with which they were able to prove rigorous bounds on what backtrack programs can do, in Computational Complexity 20 (2011), 679–740.

Exercises

▸ 1. [22 ] Explain how the tasks of generating (i) n-tuples, (ii) permutations of distinct items, (iii) combinations, (iv) integer partitions, (v) set partitions, and (vi) nested parentheses can all be regarded as special cases of backtrack programming, by presenting suitable domains Dk and cutoff properties Pl(x_1, ..., xl) that satisfy (1) and (2).

2. [10 ] True or false: We can choose D_1 so that P_1(x_1) is always true.

3. [20 ] Let T be any tree. Is it possible to define domains Dk and cutoff properties Pl(x_1, ..., xl) so that T is the backtrack tree traversed by Algorithm B?

4. [16 ] Using a chessboard and eight coins to represent queens, one can follow the steps of Algorithm B and essentially traverse the tree of Fig. 68 by hand in about three hours. Invent a trick to save half of the work.

▸ 5. [20 ] Reformulate Algorithm B as a recursive procedure called try (l), having global variables n and x_1 ... xn, to be invoked by saying ‘try (1)’. Can you imagine why the author of this book decided not to present the algorithm in such a recursive form?

6. [20 ] Given r, with 1 ≤ r ≤ 8, in how many ways can 7 nonattacking queens be placed on an 8 × 8 chessboard, if no queen is placed in row r?

7. [20 ] (T. B. Sprague, 1890.) Are there any values n > 5 for which the n queens problem has a “framed” solution with x_1 = 2, x_2 = n, xn−1 = 1, and xn = n − 1?

8. [20 ] Are there two 8-queen placements with the same x_1x_2x_3x_4x_5x_6?

9. [21 ] Can a 4m-queen placement have 3m queens on “white” squares?

▸ 10. [22 ] Adapt Algorithm W to the n queens problem, using bitwise operations on n-bit numbers as suggested in the text.

11. [M25 ] (W. Ahrens, 1910.) Both solutions of the n queens problem when n = 4 have quarterturn symmetry: Rotation by 90° leaves them unchanged, but reflection doesn’t.

1. Can the n queens problem have a solution with reflection symmetry?

2. Show that quarterturn symmetry is impossible if n mod 4 ∈ {2, 3}.

3. Sometimes the solution to an n queens problem contains four queens that form the corners of a tilted square, as shown here. Prove that we can always get another solution by tilting the square the other way (but leaving the other n − 4 queens in place).

4. Let Cn be the number of solutions with 90° symmetry, and suppose cn of them have xk > k for 1 ≤ k ≤ n / 2. Prove that Cn = 2⌊n / 4 ⌋ cn.

12. [M28 ] (Wraparound queens.) Replace (3) by the stronger conditions ‘xj ≠ xk, (xk − xj)mod n ≠ k − j, (xj − xk)mod n ≠ k − j’. (The n × n grid becomes a torus.) Prove that the resulting problem is solvable if and only if n is not divisible by 2 or 3.

13. [M30 ] For which n ≥ 0 does the n queens problem have at least one solution?

14. [M25 ] If exercise 12 has T (n) toroidal solutions, show that Q(mn) ≥ Q(m)nT (n).

15. [HM42 ] (M. Simkin, 2021.) Show that Q(n) ≈ σnn! as n → ∞, where σ ≈ 0.389068.

16. [21 ] Let H(n) be the number of ways that n queen bees can occupy an n × n honeycomb so that no two are in the same line. (For example, one of the H(4) = 7 ways is shown here.) Compute H(n) for small n.

17. [15 ] J. H. Quick (a student) noticed that the loop in step L2 of Algorithm L can be changed from ‘while xl < 0’ to ‘while xl ≠ 0’, because xl cannot be positive at that point of the algorithm. So he decided to eliminate the minus signs and just set xl+k+1 ← k in step L3. Was it a good idea?

18. [17] Suppose that n = 4 and Algorithm L has reached step L2 with l = 4 and x_1x_2x_3 = 241. What are the current values of x_4x_5x_6x_7x_8, p_0p_1p_2p_3p_4, and y_1y_2y_3?

19. [M10 ] What are the domains Dl in Langford’s problem (7)?

▸ 20. [21 ] Extend Algorithm L so that it forces xl ← k whenever k ∉ {x_1, ..., xl−1} and l ≥ 2n − k − 1.

▸ 21. [M25 ] If x = x_1x_2 ... x_2n, let xD = (−x_2n) ... (−x_2) (−x_1) = −xR be its dual.

1. Show that if n is odd and x solves Langford’s problem (7), we have xk = n for some k ≤ ⌊n/2⌋ if and only if xkD=n for some k > ⌊n / 2⌋.

2. Find a similar rule that distinguishes x from xD when n is even.

3. Consequently the algorithm of exercise 20 can be modified so that exactly one of each dual pair of solutions {x, xD} is visited.

22. [M26 ] Explore “loose Langford pairs”: Replace ‘j + k + 1’ in (7) by ‘j + ⌊3k / 2⌋’.

23. [17 ] We can often obtain one word rectangle from another by changing only a letter or two. Can you think of any 5 × 6 word rectangles that almost match (10)?

24. [20 ] Customize Algorithm B so that it will find all 5 × 6 word rectangles.

▸ 25. [25 ] Explain how to use orthogonal lists, as in Fig. 13 of Section 2.2.6, so that it’s easy to visit all 5-letter words whose kth character is c, given 1 ≤ k ≤ 5 and a ≤ c ≤ z. Use those sublists to speed up the algorithm of exercise 24.

26. [21 ] Can you find nice word rectangles of sizes 5 × 7, 5 × 8, 5 × 9, 5 × 10?

27. [22 ] What profile and average node costs replace (13) and (14) when we ask the algorithm of exercise 25 for 6 × 5 word rectangles instead of 5 × 6?

▸ 28. [23 ] The method of exercises 24 and 25 does n levels of backtracking to fill the cells of an m × n rectangle one column at a time, using a trie to detect illegal prefixes in the rows. Devise a method that does mn levels of backtracking and fills just one cell per level, using tries for both rows and columns.

29. [20 ] Do any 5 × 6 word rectangles contain fewer than 11 different words?

30. [22 ] Symmetric word squares, whose columns are the same as their rows, were popular in England during the 1850s. For example, A. De Morgan praised the square

L E A V E
E L L E N
A L O N E
V E N O M
E N E M Y

because it actually is “meaningful”! Determine the total number of symmetric 5 × 5 word squares, by adapting the method of exercise 28. How many belong to WORDS(500)?

31. [20 ] (Charles Babbage, 1864.) Do any of the symmetric 5 × 5 word squares also have valid words on both diagonals?

32. [22 ] How many symmetric word squares of sizes 2 × 2, 3 × 3, ... , are supported by The Official SCRABBLE® Players Dictionary, fourth edition (Hasbro, 2005)?

33. [21 ] Puzzlers who tried to construct word squares by hand found long ago that it was easiest to work from bottom to top. Therefore they used “reverse dictionaries,” whose words appear in colex order. Does this idea speed up computer experiments?

34. [15 ] What’s the largest commafree subset of the following words?

Click here to view code image

aced babe bade bead beef cafe cede dada dead deaf face fade feed

▸ 35. [22 ] Let w_1, w_2, ... , wn be four-letter words on an m-letter alphabet. Design an algorithm that accepts or rejects each wj, according as wj is commafree or not with respect to the accepted words of {w_1, ..., wj−1}.

36. [M22 ] A two-letter block code on an m-letter alphabet can be represented as a digraph D on m vertices, with a → b if and only if ab is a codeword.

1. Prove that the code is commafree ⇔ D has no oriented paths of length 3.

2. How many arcs can be in an m-vertex digraph with no oriented paths of length r?

▸ 37. [M30 ] (W. L. Eastman, 1965.) The following elegant construction yields a comma-free code of maximum size for any odd block length n, over any alphabet. Given a sequence x = x_0x_1 ... xn−1 of nonnegative integers, where x differs from each of its other cyclic shifts xk ... xn−1x_0 ...xk−1 for 0 < k < n, the procedure outputs a cyclic shift σx with the property that the set of all such σx is commafree.

We regard x as an infinite periodic sequence 〈xn〉 with xk = xk−n for all k ≥ n. Each cyclic shift then has the form xkxk + 1 ... xk+n−1. The simplest nontrivial example occurs when n = 3, where x = x_0x_1x_2x_0x_1x_2x_0 ... and we don’t have x_0 = x_1 = x_2. In this case the algorithm outputs xkxk+1xk+2 where xk ≥ xk+1 < xk+2; and the set of all such triples clearly satisfies the commafree condition.

One key idea is to think of x as partitioned into t substrings by boundary markers bj, where 0 ≤ b_0 < b_1 < … < bt−1 < n and bj = bj−t + n for j ≥ t. Then substring yj is xbj xbj+1 ... xbj+1−1. The number t of substrings is always odd. Initially t = n and bj = j for all j; ultimately t = 1, and σx = y_0 is the desired output.

Eastman’s algorithm is based on comparison of adjacent substrings yj−1 and yj. If those substrings have the same length, we use lexicographic comparison; otherwise we declare that the longer substring is bigger.

The second key idea is the notion of “dips,” which are substrings of the form z = z_1 ...z k where k ≥ 2 and z_1 … ≥ zk−1 < zk. It’s easy to see that any string y = y_0y_1 ... in which we have yi < yi+1 for infinitely many i can be factored into a sequence of dips, y = z(0) z(1) ... , and this factorization is unique. For example,

3141592653589793238462643383 . . . = 314 15 926 535 89 79 323 846 26 4338 3 . . . .

Furthermore, if y is a periodic sequence, its factorization into dips is also ultimately periodic, although some of the initial factors may not occur in the period. For example,

123443550123443550123443550 . . . = 12 34 435 501 23 4435 501 23 4435 . . . .

Given a periodic, nonconstant sequence y described by boundary markers as above, where the period length t is odd, its periodic factorization will contain an odd number of odd-length dips. Each round of Eastman’s algorithm simply retains the boundary points at the left of those odd-length dips. Then t is reset to the number of retained boundary points, and another round begins if t > 1.

1. Play through the algorithm by hand when n = 19 and x = 3141592653589793238.

2. Show that the number of rounds is at most ⌊log_3 n⌋.

3. Exhibit a binary x that achieves this worst-case bound when n = 3e.

4. Implement the algorithm with full details. (It’s surprisingly short!)

5. Explain why the algorithm yields a commafree code.

38. [HM28] What is the probability that Eastman’s algorithm finishes in one round? (Assume that x is a random m-ary string of odd length n > 1, unequal to any of its other cyclic shifts. Use a generating function to express the answer.)

39. [18] Why can’t a commafree code of length (m^4 − m^2)/4 contain 0001 and 2000?

▸ 40. [15] Why do you think sequential data structures such as (16)−(23) weren’t featured in Section 2.2.2 of this series of books (entitled “Sequential Allocation”)?

41. [17] What’s the significance of (a) MEM[40d]=5e and (b) MEM[904]=84 in Table 1?

42. [18] Why does Table 2 have (a) MEM[f8] = e7 and (b) MEM[a0d] = ba?

43. [20] Suppose you’re using the undoing scheme (26) and the operation σ ← σ +1 has just bumped the current stamp σ to zero. What should you do?

▸ 44. [25] Spell out the low-level implementation details of the candidate selection process in step C2 of Algorithm C. Use the routine store(a, v) of (26) whenever changing the contents of MEM, and use the following selection strategy:

1. Find a class c with the least number r of blue words.

2. If r = 0, set x ← −1; otherwise set x to a word in class c.

3. If r > 1, use the poison list to find an x that maximizes the number of blue words that could be killed on the other side of the prefix or suffix list that contains x.

▸ 45. [28] Continuing exercise 44, spell out the details of step C3 when x ≥ 0.

1. What updates should be done to MEM when a blue word x becomes red?

2. What updates should be done to MEM when a blue word x becomes green?

3. Step C3 begins its job by making x green as in part (b). Explain how it should finish its job by updating the poison list.

46. [M35] Is there a binary (m = 2) commafree code with one codeword in each of the (Σd\n μ(d)2n/d)/n cycle classes, for every word length n?

47. [HM29] A commafree code on m letters is equivalent to at most 2m! such codes if we permute the letters and/or replace each codeword by its left-right reflection.

Determine all of the nonisomorphic commafree codes of length 4 on m letters when m is (a) 2 (b) 3 (c) 4 and there are (a) 3 (b) 18 (c) 57 codewords.

48. [M42] Find a maximum-size commafree code of length 4 on m = 5 letters.

49. [20] Explain how the choices in Fig. 69 were determined from the “random” bits that are displayed. For instance, why was X_2 set to 1 in Fig. 69(b)?

50. [M15] Interpret the value E(D_1 ... Dl), in the text’s Monte Carlo algorithm.

51. [M22] What’s a simple martingale that corresponds to Theorem E?

▸ 52. [HM25] Elmo uses Algorithm E with Dk = {1,..., n}, Pl = [x_1 > ... >xl], c = 1.

1. Alice flips n coins independently, where coin k yields “heads” with probability 1/k. True or false: She obtains exactly l heads with probability [nl]/n!.

2. Let Y_1, Y_2, ... , Yl be the numbers on the coins that come up heads. (Thus Y_1 = 1, and Y_2 = 2 with probability 1/2.) Show that Pr(Alice obtains Y_1, Y_2, ... , Yl) = Pr(Elmo obtains X_1 = Yl, X_2 = Yl−1, ... , Xl = Y_1).

3. Prove that Alice q.s. obtains at most (ln n)(ln ln n) heads.

4. Consequently Elmo’s S is q.s. less than exp((ln n)2(ln ln n)).

▸ 53. [M30] Extend Algorithm B so that it also computes the minimum, maximum, mean, and variance of the Monte Carlo estimates S produced by Algorithm E.

54. [M21] Instead of choosing each yi in step E5 with probability 1/d, we could use a biased distribution where Pr(I = i | X_1,..., Xl−1) = pX_1... Xl−1(yi) > 0. How should the estimate S be modified so that its expected value in this general scheme is still C()?

55. [M20] If all costs c(x_1,..., xl) are positive, show that the biased probabilities of exercise 54 can be chosen in such a way that the estimate S is always exact.

▸ 56. [M25] The commafree code search procedure in Algorithm C doesn’t actually fit the mold of Algorithm E, because it incorporates lookahead, dynamic ordering, reversible memory, and other enhancements to the basic backtrack paradigms. How could its running time be reliably estimated with Monte Carlo methods?

57. [HM21] Algorithm E can potentially follow M different paths X_1 ... Xl−1 before it terminates, where M is the number of leaves of the backtrack tree. Suppose the final values of D at those leaves are D(1), ... , D(M). Prove that (D(1) ... D(M))1/M ≥ M.

58. [27] The text’s king path problem is a special case of the general problem of counting simple paths from vertex s to vertex t in a given graph.

We can generate such paths by random walks from s that don’t get stuck, if we maintain a table of values DIST(v) for all vertices v not yet in the path, representing the shortest distance from v to t through unused vertices. For with such a table we can simply move at each step to a vertex for which DIST(v) < ∞.

Devise a way to update the DIST table dynamically without unnecessary work.

59. [26] A ZDD with 3,174,197 nodes can be constructed for the family of all simple corner-to-corner king paths on a chessboard, using the method of exercise 7.1.4−225. Explain how to use this ZDD to compute (a) the total length of all paths; (b) the number of paths that touch any given subset of the center and/or corner points.

▸ 60. [20] Experiment with biased random walks (see exercise 54), weighting each non-dead-end king move to a new vertex v by 1 + DIST(v)2 instead of choosing every such move with the same probability. Does this strategy improve on Fig. 70?

61. [HM26] Let Pn be the number of integer sequences x_1 ... xn such that x_1 = 1 and 1 ≤ xk+1 ≤ 2xk for 1 ≤ k < n. (The first few values are 1, 2, 6, 26, 166, 1626, ... ; this sequence was introduced by A. Cayley in Philosophical Magazine (4) 13 (1857), 245−248, who showed that Pn enumerates the partitions of 2n − 1 into powers of 2.)

1. Show that Pn is the number of different profiles that are possible for a binary tree of height n.

2. Find an efficient way to compute Pn for large n. Hint: Consider the more general sequence Pn(m), defined similarly but with x_1 = m.

3. Use the estimation procedure of Theorem E to prove that Pn≥2(n2)/(n−1)!.

▸ 62. [22] When the faces of four cubes are colored randomly with four colors, estimate the probability that the corresponding “Instant Insanity” puzzle has a unique solution. How many 2-regular graphs tend to appear during the “factored” solution process?

63. [20] Find five cubes, each of whose faces has one of five colors, and where every color occurs at least five times, such that the corresponding puzzle has a unique solution.

64. [24] Assemble five cubes with uppercase letters on each face, using the patterns

By extending the principles of “Instant Insanity,” show that these cubes can be placed in a row so that four 5-letter words are visible. (Each word’s letters should have a consistent orientation. The letters C and U, H and I, N and Z are related by 90° rotation.)

65. [25] Show that the generalized “Instant Insanity” problem, with n cubes and n colors on their faces, is NP-complete, even though cases with small n are fairly easy.

▸ 66. [23] (The Fool’s Disk.) “Rotate the four disks of the left-hand illustration below so that the four numbers on each ray sum to 12.” (The current sums are 4 + 3 + 2 + 4 = 13, etc.) Show that this problem factors nicely, so that it can be solved readily by hand.

▸ 67. [26] (The Royal Aquarium Thirteen Puzzle.) “Rearrange the nine cards of the right-hand illustration above, optionally rotating some of them by 180°, so that the six horizontal sums of gray letters and the six vertical sums of black letters all equal 13.” (The current sums are 1 + 5 + 4 = 10, ... , 7 + 5 + 7 = 19.) The author of Hoffmann’s Puzzles Old and New (1893) stated that “There is no royal road to the solution. The proper order must be arrived at by successive transpositions until the conditions are fulfilled.” Prove that he was wrong: “Factor” this problem and solve it by hand.

▸ 68. [28] (Johan de Ruiter, 14 March 2018.) Put a digit into each empty box, in such a way that every box names the exact number of distinct digits that it points to.

69. [41] Is there a puzzle like exercise 68 whose clues contain more than 32 digits of π?

70. [HM40] (M. Bousquet-Mélou.) Consider self-avoiding paths from the upper left corner of an m × n grid to the lower right, where each step is either up, down, or to the right. If we generate such paths at random, making either 1 or 2 or 3 choices at each step as in Algorithm E, the expected value E Dmn is the total number of such paths, mn−1. But the variance is considerably larger: Construct polynomials Pm(z) and Qm(z) such that we have Gm(z)  =  ∑n=1∞(E Dmn2)zn⁢= zPm(z)/Qm(z) for m ≥ 2. For example, G_3(z) = (z + z^2)/(1–9z –6z^2) = z+ 10z^2 + 96z^3 + 924z^4 + 8892z^5 + . . . . Prove furthermore that EDmn2=Θ(pmn), where ρm = 2m + O(1).

Table 666
TWENTY QUESTIONS (SEE EXERCISE 71)

1.

The first question whose answer is A is:

(A) 1

(B) 2

(C) 3

(D) 4

(E) 5

2.

The next question with the same answer as this one is:

(A) 4

(B) 6

(C) 8

(D) 10

(E) 12

3.

The only two consecutive questions with identical answers are questions:

(A) 15 and 16

(B) 16 and 17

(C) 17 and 18

(D) 18 and 19

(E) 19 and 20

4.

The answer to this question is the same as the answers to questions:

(A) 10 and 13

(B) 14 and 16

(C) 7 and 20

(D) 1 and 15

(E) 8 and 12

5.

The answer to question 14 is:

(A) B

(B) E

(C) C

(D) A

(E) D

6.

The answer to this question is:

(A) A

(B) B

(C) C

(D) D

(E) none of those

7.

An answer that appears most often is:

(A) A

(B) B

(C) C

(D) D

(E) E

8.

Ignoring answers that appear equally often, the least common answer is:

(A) A

(B) B

(C) C

(D) D

(E) E

9.

The sum of all question numbers whose answers are correct and the same as this one is:

(A) ∈ [59 . . 62]

(B) ∈ [52 . . 55]

(C) ∈ [44 . . 49]

(D) ∈ [59 . . 67]

(E) ∈ [44 . . 53]

10.

The answer to question 17 is:

(A) D

(B) B

(C) A

(D) E

(E) wrong

11.

The number of questions whose answer is D is:

(A) 2

(B) 3

(C) 4

(D) 5

(E) 6

12.

The number of other questions with the same answer as this one is the same as the number of questions with answer:

(A) B

(B) C

(C) D

(D) E

(E) none of those

13.

The number of questions whose answer is E is:

(A) 5

(B) 4

(C) 3

(D) 2

(E) 1

14.

No answer appears exactly this many times:

(A) 2

(B) 3

(C) 4

(D) 5

(E) none of those

15.

The set of odd-numbered questions with answer A is:

(A) {7}

(B) {9}

(C) not {11}

(D) {13}

(E) {15}

16.

The answer to question 8 is the same as the answer to question:

(A) 3

(B) 2

(C) 13

(D) 18

(E) 20

17.

The answer to question 10 is:

(A) C

(B) D

(C) B

(D) A

(E) correct

18.

The number of prime-numbered questions whose answers are vowels is:

(A) prime

(B) square

(C) odd

(D) even

(E) zero

19.

The last question whose answer is B is:

(A) 14

(B) 15

(C) 16

(D) 17

(E) 18

20.

The maximum score that can be achieved on this test is:

(A) 18

(B) 19

(C) 20

(D) indeterminate

(E) achievable only by getting this question wrong

▸ 71. [M29] (Donald R. Woods, 2000.) Find all ways to maximize the number of correct answers to the questionnaire in Table 666. Each question must be answered with a letter from A to E. Hint: Begin by clarifying the exact meaning of this exercise. What answers are best for the following two-question, two-letter “warmup problem”?

1. (A) Answer 2 is B.

(B) Answer 1 is A.

2. (A) Answer 1 is correct.

(B) Either answer 2 is wrong or answer 1 is A, but not both.

72. [HM28] Show that exercise 71 has a surprising, somewhat paradoxical answer if two changes are made to Table 666: 9(E) becomes ‘∈ [39 .. 43]’; 15 (C) becomes ‘{11}’.

▸ 73. [30] (A clueless anacrostic.) The letters of 29 five-letter words

12345⁢,678910,1112131415, 1617181920⁢,.⁢ .⁢ .⁢ ,141142143144145,

all belonging to WORDS(1000), have been shuffled to form the following mystery text:

30299⁢  1401213145904599⁢  26⁢107⁢  478453512713339⁢  137139⁢  661126914⁢820911297016⁢7931985⁢  1017678441010660⁢  118119⁢  2425100⁢  15641171⁢  42122123103104634931121987980⁢  ⁢4648⁢  134135131  1439614212050132334340.⁢            .⁢ .⁢ .⁢ 11197113105381026265114⁢  74828183136372161888655⁢  (3235⁢  ⁢11711623535617189467⁢  12815575889⁢  87109⁢  24628953126771445441⁢)⁢  68115751387312436130127141⁢  2292⁢  7259⁢  108125⁢110.

Furthermore, their initial letters 1,6,11,16,.⁢.⁢.⁢ ,141 identify the source of that quotation, which consists entirely of common English words. What does it say?

74. [21] The fifteenth mystery word in exercise 73 is ‘134135131’. Why does its special form lead to a partial factorization of that problem?

▸ 75. [30] (Connected subsets.) Let v be a vertex of some graph G, and let H be a connected subset of G that contains v. The vertices of H can be listed in a canonical way by starting with v_0 ← v and then letting v_1, v_2, ... be the neighbors of v_0 that lie in H, followed by the neighbors of v_1 that haven’t yet been listed, and so on. (We assume that the neighbors of each vertex are listed in some fixed order.)

For example, if G is the 3 × 3 grid P_3 □ P_3, exactly 21 of its connected five-element subsets contain the upper left corner element v. Their canonical orderings are

if we order the vertices top-to-bottom, left-to-right when listing their neighbors. (Vertices labeled ⓪, ①, ②, ③, ④ indicate ν_0, ν_1, ν_2, ν_3, ν_4. Other vertices are not in H.)

Design a backtrack algorithm to generate all of the n-element connected subsets that contain a specified vertex v, given a graph that is represented in SGB format (which has ARCS, TIP, and NEXT fields, as described near the beginning of Chapter 7).

76. [23] Use the algorithm of exercise 75 to generate all of the connected n-element subsets of a given graph G. How many such subsets does Pn □ Pn have, for 1 ≤ n ≤ 9?

77. [M22] A v-reachable subset of a directed graph G is a nonempty set of vertices H with the property that every u ∈ H can be reached from v by at least one oriented path in G | H. (In particular, v itself must be in H.)

1. The digraph P3→□P3→ is like P_3 □ P_3, except that all arcs between vertices are directed downward or to the right. Which of the 21 connected subsets in exercise 75 are also v-reachable from the upper left corner element v of P3→□P3→?

2. True or false: H is v-reachable if and only if G | H contains a dual oriented spanning tree rooted at v. (An oriented tree has arcs u → pu, where pu is the parent of the nonroot node u; in a dual oriented tree, the arcs are reversed: pu → u.)

3. True or false: If G is undirected, so that w → u whenever u → w, its v-reachable subsets are the same as the connected subsets that contain v.

4. Modify the algorithm of exercise 75 so that it generates all of the n-element v-reachable subsets of a digraph G, given n, v, and G.

78. [22] Extend the algorithm of exercise 77 to weighted graphs, in which every vertex has a nonnegative weight: Generate all of the connected induced subgraphs whose total weight w satisfies L ≤ w < U.

▸ 79. [M30] The author and his wife own a pipe organ that contains 812 pipes, each of which is either playing or silent. Therefore 2^812 different sounds (including silence) can potentially be created. However, the pipes are controlled by a conventional organ console, which has only 56+56+32 = 144 keys and pedals that can be played by hands and feet, together with 20 on-off switches to define the connections between keys and pipes. Therefore at most 2^164 different sounds are actually playable! The purpose of this exercise is to determine the exact number of n-pipe playable sounds, for small n.

The keys are binary vectors s = s_0s_1 ...s_55 and g = g_0g_1 ...g_55; the pedals are p = p_0p_1 ... p_31; the console control switches are c = c_0c_1 ... c_19; and the pipes are ri,j for 0 ≤ i < 16 and 0 ≤ j < 56. Here are the precise rules that define the pipe activity ri,j in terms of the input vectors s, g, p, and c that are governed by the organist:

ri,j={cipj⁢ v⁢ ci+15pj−12,i∈{0,1};cipj,i∈{2};ri,j={(ci⁢v⁢ ci+1[j<12])sj*,i∈{3};ci[j≥12]sj*,i∈{4,8};ci⁢sj*,i∈{5,6,7};ri,j={(ci∨ci+1[j<12])gj*,i∈{9};ci[j⁢ ≥12]gj*,i∈{10};ci⁢ gj*,i∈{11,12};(c13∨c14)gj*,i∈{13};c14⁢gj*.i∈{14,15}.

Here pj = 0 if j < 0 or j ≥ 32; sj*=sj∨c17gj; gj*=gj⁢ ∨c19pj. [In organ jargon, the array of pipes has 16 “ranks”; ranks {0, 1, 2}, {3,..., 8}, {9,..., 15} constitute the Pedal, Swell, and Great divisions. Ranks 3 and 4 share their lower 12 pipes, as do ranks 9 and 10. Ranks 13, 14, and 15 form a “mixture,” c_14. Unit ranks c_15 and c_16 extend ranks 0 and 1, twelve notes higher. Console switches c_17, c_18, c_19 are “couplers” Swell → Great, Swell → Pedal, Great → Pedal, which explain the formulas for sj* and gj*.]

A playable sound S is a set of pairs (i, j) such that we have ri,j = [(i, j) ∈ S] for at least one choice of the input vectors s, g, p, c. For example, the first chord of Bach’s Toccata in D minor is the 8-pipe sound {(3, 33), (3, 45), (4, 33), (4, 45), (5, 33), (5, 45), (6, 33), (6, 45)}, which is achievable when s_33 = s_45 = c_3 = c_4 = c_5 = c_6 = 1 and all other inputs are 0. We want to find the number Qn of playable sounds with ॥S॥ = n.

1. There are 16 × 56 variables ri,j but only 812 actual pipes, because some of the ranks are incomplete. For which pairs (i, j) is ri,j always false?

2. True or false: If s ⊆ s′, g ⊆ g′, p ⊆ p′, and c ⊆ c′, then r ⊆ r′.

3. Show that every playable sound is achievable with c_17 = c_18 = c_19 = 0.

4. Find a 5-pipe playable sound in which just five of the sj, gj, pj, cj are nonzero.

5. For which i and i′ are the 2-pipe sounds {(i, 40), (i′, 50)} playable?

6. Determine Q_1 by hand, and explain why it is less than 812.

7. Determine Q_811 by hand.

8. Determine Q_2, ... , Q_10 by computer, and compare them to (8122),.⁢ ⁢.⁢ ⁢.⁢ ⁢,(81210).

We hold several threads in our hands,
and the odds are that one or other of them guides us to the truth.

We may waste time in following the wrong one,
but sooner or later we must come upon the right.

— SHERLOCK HOLMES, in The Hound of the Baskervilles (1901)

The following Receipts are not a mere marrow-less collection of
shreds, and patches, and cuttings, and pastings, from obsolete works,
but a bona fide register of practical facts . . .

the author submitting to a labour no preceding cookery-book-maker, perhaps,
ever attempted to encounter; and having not only dressed, but eaten
each Receipt before he set it down in his book.

— WILLIAM KITCHINER, Apicius Redivivus; Or, The Cook’s Oracle (1817)

Just as we hope you will learn from us, we have learned from you,
from the recipes and short cuts and tips and traditions
you have been kind enough to tell us about.

Without your help, truly, this book could not have been written.

— McCall’s Cook Book (1963)

What a dance
do they do
Lordy, how I’m tellin’ you!

— HARRY BARRIS, Mississippi Mud (1927)

Don’t lose your confidence if you slip,
Be grateful for a pleasant trip,
And pick yourself up, dust yourself off, start all over again.

— DOROTHY FIELDS, Pick Yourself Up (1936)

7.2.2.1. Dancing links

One of the chief characteristics of backtrack algorithms is the fact that they usually need to undo everything that they do to their data structures. In this section we’ll study some extremely simple link-manipulation techniques that modify and unmodify the structures with ease. We’ll also see that these ideas have many, many practical applications.

Suppose we have a doubly linked list, in which each node X has a predecessor and successor denoted respectively by LLINK(X) and RLINK(X). Then we know that it’s easy to delete X from the list, by setting

RLINK(LLINK(X)) ←RLINK(X),LLINK (RLINK(X)) ← LLINK(X).(1)

At this point the conventional wisdom is to recycle node X, making it available for reuse in another list. We might also want to tidy things up by clearing LLINK(X) and RLINK(X) to ∧, so that stray pointers to nodes that are still active cannot lead to trouble. (See, for example, Eq. 2.2.5–(4), which is the same as (1) except that it also says ‘AVAIL ⇐ X’.) By contrast, the dancing-links trick resists any urge to do garbage collection. In a backtrack application, we’re better off leaving LLINK(X) and RLINK(X) unchanged. Then we can undo operation (1) by simply setting

RLINK(LLINK(X)) ←X,LLINK (RLINK(X)) ← X.(2)

For example, we might have a 4-element list, as in 2.2.5–(2):

If we use (1) to delete the third element, (3) becomes

And if we now decide to delete the second element also, we get

Subsequent deletion of the final element, then the first, will leave us with this:

The list is now empty, and its links have become rather tangled. (See exercise 1.) But we know that if we proceed to backtrack at this point, using (2) to undelete elements 1, 4, 2, and 3 in that order, we will magically restore the initial state (3). The choreography that underlies the motions of these pointers is fun to watch, and it explains the name “dancing links.”

Exact cover problems

We will be seeing many examples where links dance happily and efficiently, as we study more and more examples of backtracking. The beauty of the idea can perhaps be seen most naturally in an important class of problems known as exact covering: We’re given an M × N matrix A of 0s and 1s, and the problem is to find all subsets of rows whose sum is exactly 1 in every column. For example, consider the 6 × 7 matrix

A=(001010010010010110010100101001000010001101)(5)

Each row of A corresponds to a subset of a 7-element universe. A moment’s thought shows that there’s only one way to cover all seven of these columns with disjoint rows, namely by choosing rows 1, 4, and 5. We want to teach a computer how to solve such problems, when there are many, many rows and many columns.

Matrices of 0s and 1s appear frequently in combinatorial problems, and they help us to understand the relations between problems that are essentially the same although they appear to be different (see exercise 5). But inside a computer, we rarely want to represent an exact cover problem explicitly as a two-dimensional array of bits, because the matrix tends to be extremely sparse: There normally are very few 1s. Thus we’ll use a different representation, essentially with one node in our data structure for each 1 in the matrix.

Furthermore, we won’t even talk about rows and columns! Some of the exact cover problems we deal with already involve concepts that are called “rows” and “columns” in their own areas of application. Instead we will speak of options and items: Each option is a set of items; and the goal of an exact cover problem is to find disjoint options that cover all the items.

For example, we shall regard (5) as the specification of six options involving seven items. Let’s name the items a, b, c, d, e, f, g; then the options are

‘c⁢ e’;‘a⁢ d⁢ g’;‘b⁢ c⁢ f’;‘a⁢ d⁢ f’;‘b⁢ g’;‘d⁢ e⁢ g’.(6)

The first, fourth, and fifth options give us each item exactly once.

One of the nicest things about exact cover problems is that every tentative choice we make leaves us with a residual exact cover problem that is smaller — often substantially smaller. For example, suppose we try to cover item a in (6) by choosing the option ‘a d g’: The residual problem has only two options,

‘c⁢ e’and‘b⁢ c⁢ f’,(7)

because the other four involve the already-covered items. Now it’s easy to see that (7) has no solution; therefore we can remove option ‘a d g’ from (6). That leaves us with only one option for item a, namely ‘a d f’. And its residual,

‘c⁢ e’and‘b⁢ g’,(8)

gives us the solution we were looking for.

Thus we’re led naturally to a recursive algorithm that’s based on the primitive operation of “covering an item”: To cover item i, we delete all of the options that involve i, from our database of currently active options, and we also delete i from the list of items that need to be covered. The algorithm is then quite simple.

•Select an item i⁢  that needs to be covered; but terminate⁢         successfully if none are left (we've found a solution).•If no active option involve i, terminate unsuccessfully        (there's no solution).⁢  Otherwise cover item i.•For each just-deleted option O⁢ that involves i, one at a time, do this:         for each item j≠i in O, cover item j⁢ ;then         recursively append O to each solution of the residual problem.(9)

(Everything that’s covered must later be uncovered, of course, as we’ll see.)

Interesting details arise when we flesh out this algorithm and look at appropriate low-level mechanisms. There’s a doubly linked “horizontal” list of all items that need to be covered; and each item also has its own “vertical” list of all the active options that involve it. For example, the data structures for (6) are:

(In this diagram, doubly linked pointers “wrap around” at the dotted lines.) The horizontal list has LLINK and RLINK pointers; the vertical lists have ULINK and DLINK. Nodes of each vertical list also point to their list header via TOP fields.

The top row of diagram (10) shows the initial state of the horizontal item list and its associated vertical headers. The other rows illustrate the six options of (6), which are represented by sixteen nodes within the vertical lists. Those options implicitly form horizontal lists, indicated by light gray lines; but their nodes don’t need to be linked together with pointers, because the option lists don’t change. We can therefore save time and space by allocating them sequentially. On the other hand, our algorithm does require an ability to traverse each option cyclically, in both directions. Therefore we insert spacer nodes between options. A spacer node x is identified by the condition TOP(x) ≤ 0; it also has

ULINK(x) =  address of the first node in the option before x;ULINK(x) =  address of the last node in the option after x.(11)

These conventions lead to the internal memory layout shown in Table 1. First come the records for individual items; those records have NAME, LLINK, and RLINK fields, where NAME is used in printouts. Then come the nodes, which have TOP, ULINK, and DLINK fields. The TOP field is, however, called LEN in the nodes that serve as item headers, because Algorithm X below uses those fields to store the lengths of the item lists. Nodes 8, 11, 15, 19, 23, 26, and 30 in this example are the spacers. Fields marked ‘—’ are unused and can contain anything.

Table 1
THE INITIAL CONTENTS OF MEMORY CORRESPONDING TO (6) AND (10)

i:

0

1

2

3

4

5

6

7

NAME(i):

—

a

b

c

d

e

f

g

LLINK(i):

7

0

1

2

3

4

5

6

RLINK(i):

1

2

3

4

5

6

7

0

x:

0

1

2

3

4

5

6

7

LEN(x):

—

2

2

2

3

2

2

3

ULINK(x):

—

20

24

17

27

28

22

29

DLINK(x):

—

12

16

9

13

10

18

14

x:

8

9

10

11

12

13

14

15

TOP(x):

0

3

5

−1

1

4

7

−2

ULINK(x):

—

3

5

9

1

4

7

12

DLINK(x):

10

17

28

14

20

21

25

18

x:

16

17

18

19

20

21

22

23

TOP(x):

2

3

6

−3

1

4

6

−4

ULINK(x):

2

9

6

16

12

13

18

20

DLINK(x):

24

3

22

22

1

27

6

25

x:

24

25

26

27

28

29

30

TOP(x):

2

7

−5

4

5

7

−6

ULINK(x):

16

14

24

21

10

25

27

DLINK(x):

2

29

29

4

5

7

—

OK, we’re ready now to spell out precisely what happens inside the computer’s memory when Algorithm X wants to cover a given item i:

cover(i) = {Set  p←DLINK(i).   (Undeclared variables like p,l,r⁢  are local.)While p≠i⁢, hide(p),  then set p←DLINK(P)  and repeat.Set  l←LLINK(i),  r←RLINK(i),        RLINK(i)   ←r,  LLINK(r)  ← l.(12)

hide(p) = {Set q⁢ ←p+1,and repeat the following while q⁢ ≠p:    Set x←TOP(q),  u←ULINK(q), d←DLINK(q);    if x⁢  ≤0,set q←u⁢ (q⁢ was a spacer);⁢    otherwise set DLINK(u) ←d, ULINK(d)←u,           LEN(x)←LEN(x)−1, q←q+1.  (13)

And — here’s the point — those operations can readily be undone:

uncover(i)   = {Set   l←LLINK(i),  r←RLINK(i)            RLINK (l)←i,  LLINK(r)←i.Set  p←ULINK(i).While  p≠i,unhide(p), than set p←ULINK(p) and repeat.(14)

unhide(p)={Setq←p−1,and repeat the following while q≠p:Set x←TOP(q), u←ULINK(q), d←DLINK(q);if x≤0,set q←d⁢(q⁢   was a spaer);otherwise set DLINK(u)←q, ULINK(d) ← q,            LEN (x) ←LEN (x) +1, q←q−1.(15)

We’re careful here to do everything backwards, using operation (2) inside (14) and (15) to undelete in precisely the reverse order of the way that we’d previously used operation (1) inside (12) and (13) to delete. Furthermore, we’re able to do this in place, without copying, by waltzing through the data structure at the same time as we’re modifying it.

Algorithm X (Exact cover via dancing links). This algorithm visits all solutions to a given exact cover problem, using the data structures just described. It also maintains a list x_0, x_1, ..., xT of node pointers for backtracking, where T is large enough to accommodate one entry for each option in a partial solution.

X1. [Initialize.] Set the problem up in memory, as in Table 1. (See exercise 8.) Also set N to the number of items, Z to the last spacer address, and l ← 0.

X2. [Enter level l.] If RLINK(0) = 0 (hence all items have been covered), visit the solution that is specified by x_0x_1 . . . xl−1 and go to X8. (See exercise 13.)

X3. [Choose i.] At this point the items i_1, . . . , it still need to be covered, where i_1 = RLINK(0), ij+1 = RLINK(ij), RLINK(it) = 0. Choose one of them, and call it i. (The MRV heuristic of exercise 9 often works well in practice.)

X4. [Cover i.] Cover item i using (12), and set xl ← DLINK(i).

X5. [Try xl.] If xl = i, go to X7 (we’ve tried all options for i). Otherwise set p ← xl + 1, and do the following while p ≠ xl: Set j ← TOP(p); if j ≤ 0, set p ← ULINK(p); otherwise cover(j) and set p ← p + 1. (This covers the items ≠ i in the option that contains xl.) Set l ← l + 1 and return to X2.

X6. [Try again.] Set p ← xl − 1, and do the following while p ≠ xl: Set j ← TOP(p); if j ≤ 0, set p ← DLINK(p); otherwise uncover(j) and set p ← p − 1. (This uncovers the items ≠ i in the option that contains xl, using the reverse of the order in X5.) Set i ← TOP(xl), xl ← DLINK(xl), and return to X5.

X7. [Backtrack.] Uncover item i using (14).

X8. [Leave level l.] Terminate if l = 0. Otherwise set l ← l − 1 and go to X6.

The reader is strongly advised to work exercise 11 now — yes, now, really! — in order to experience the dance steps of this instructive algorithm. When the procedure terminates, all of the links will be restored to their original settings.

We’re going to see lots of applications of Algorithm X, and similar algorithms, in this section. Let’s begin by fulfilling a promise that was made on page 2 of Chapter 7, namely to solve the problem of Langford pairs efficiently by means of dancing links.

The task is to put 2n numbers {1, 1, 2, 2, ..., n, n} into 2n slots s_1 s_2 ... s2n, in such a way that exactly i numbers fall between the two occurrences of i. It illustrates exact covering nicely, because we can regard the n values of i and the 2n slots sj as items to be covered. The allowable options for placing the i’s are then

‘i⁢  sj⁢ sk’,for 1≤j⁢ <k≤2n,k=i+j+1,1≤i≤n;(16)

for example, when n = 3 they’re

‘1⁢ s1⁢ s3’, ‘1s2 s4’,⁢   ‘1s3 s5’, ‘1s4 s6’,⁢  ‘2s1 s4’, ‘2s2 s5’, ‘2s3 s6’, ‘3s1 s5’,⁢  ‘3s2 s6’.(17)

An exact covering of all items is equivalent to placing each pair and filling each slot. Algorithm X quickly determines that (17) has just two solutions,

‘3 s_1 s_5’, ‘2 s_3 s_6’, ‘1 s_2 s_4’   and   ‘3 s_2 s_6’, ‘2 s_1 s_4’, ‘1 s_3 s_5’,
corresponding to the placements 3 1 2 1 3 2 and 2 3 1 2 1 3. Notice that those placements are mirror images of each other; exercise 15 shows how to save a factor of 2 and eliminate such symmetry, by omitting some of the options in (16).

With that change, there are exactly 326,721,800 solutions when n = 16, and Algorithm X needs about 1.13 trillion memory accesses to visit them all. That’s pretty good — it amounts to roughly 3460 mems per solution, as the links whirl.

Of course, we’ve already looked at a backtrack procedure that’s specifically tuned to the Langford problem, namely Algorithm 7.2.2L near the beginning of Section 7.2.2. With the enhancement of exercise 7.2.2–21, that one handles the case n = 16 somewhat faster, finishing after about 400 billion mems. But Algorithm X can be pleased that its general-purpose machinery isn’t way behind the best custom-tailored method.

Secondary items

Can the classical problem of n queens also be formulated as an exact cover problem? Yes, of course! But the construction isn’t quite so obvious. Instead of setting the problem up as we did in 7.2.2–(3), where we chose a queen placement for each row of the board, we shall now allow both rows and columns to participate equally when making the necessary choices.

There are n^2 options for placing queens, and we want exactly one queen in every row and exactly one in every column. Furthermore, we want at most one queen in every diagonal. More precisely, if xij is the binary variable that signifies a queen in row i and column j, we want

∑i=1n⁢xij=1for 1≤j⁢≤n;∑j=1n⁢xij=1for 1≤i≤n;(18)

∑{xij⁢| 1≤i,j≤n, i+j=s}≤1for 1 < s ≤2n;(19)

∑{xij⁢| 1≤i,j≤n, i−j=d}≤1for −n < d⁢< n.(20)

The inequalities in (19) and (20) can be changed to equalities by introducing “slack variables” u_2, ..., u2n, υ−n+1, ..., υn−1, each of which is 0 or 1:

∑{xij⁢| 1≤i,j≤n, i+j=s}+us=1for 1<s≤2 n;(21)

∑{xij⁢| 1≤i,j≤n, i−j=d}+ud=1for − n<d< n.(22)

Thus we’ve shown that the problem of n nonattacking queens is equivalent to the problem of finding n^2 + 4n − 2 binary variables xij, us, υd for which certain subsets of the variables sum to 1, as specified in (18), (21), and (22).

And that is essentially an exact cover problem, whose options correspond to the binary variables and whose items correspond to the subsets. The items are ri, cj, as, and bd, representing respectively row i, column j, upward diagonal s, and downward diagonal d. The options are ‘ri cj ai+j bi−j’ for queen placements, together with trivial options ‘as’ and ‘bd’ to take up any slack.

For example, when n = 4 the n^2 placement options are

‘r1⁢⁢ c1 a2⁢  b0’;‘r1⁢⁢ c2 a3⁢  b− 1’;‘r1⁢⁢ c3⁢⁢ a4 b−2⁢’;‘r1⁢⁢ c4⁢⁢ a5 b−3⁢’;‘r2⁢⁢ c1 a3⁢  b1’;‘r2⁢⁢ c2 a4⁢  b0’;‘r2⁢⁢ c3⁢⁢ a5 b−1⁢’;‘r2⁢⁢ c4⁢⁢ a6 b−2⁢’;‘r3⁢⁢ c1 a4⁢  b2’;‘r3⁢⁢ c2 a5⁢  b 1’;‘r3⁢⁢ c3⁢⁢ a6 b0⁢’;’r3⁢⁢ c4⁢⁢ a7 b−1⁢’;’r4⁢⁢ c1 a5⁢  b3’;’r4⁢⁢ c2 a6⁢  b2’;’r4⁢⁢ c3⁢⁢ a7 b1⁢’;‘r4⁢⁢ c4⁢⁢ a8 b0⁢’;(23)

and the 4n − 2 slack options (which contain just one item each) are

‘a2’;‘a3’;‘a4’;‘a5’;‘a6’;‘a7’;‘a8’;‘b−3’;‘b−2’;‘b−1’;‘b0’;‘b1’;‘b2’;‘b3’; (24)

Algorithm X will solve this small problem easily, although its treatment of the slacks is somewhat awkward (see exercise 16).

A closer look shows, however, that a slight change to Algorithm X will allow us to avoid slack options entirely! Let’s divide the items of an exact cover problem into two groups: primary items, which must be covered exactly once, and secondary items, which must be covered at most once. If we simply modify step X1 so that only the primary items appear in the active list, everything will work like a charm. (Think about it.) In fact, the necessary changes to step X1 already appear in the answer to exercise 8.

Secondary items turn out to be extremely useful in applications. So let’s redefine the exact cover problem, taking them into account: Henceforth we shall assume that an exact cover problem involves N distinct items, of which N_1 are primary and N_2 = N − N_1 are secondary. It is defined by a family of options, each of which is a subset of the items. Every option must include at least one primary item. The task is to find all subsets of the options that (i) contain every primary item exactly once, and (ii) contain every secondary item at most once.

(Options that are purely secondary are excluded from this new definition, because they will never be chosen by Algorithm X as we’ve refined it. If for some reason you don’t like that rule, you can always go back to the idea of slack options. Exercise 19 discusses another interesting alternative.)

The order in which primary items appear in Algorithm X’s active list can have a significant effect on the running time, because the implementation of step X3 in exercise 9 selects the first item of minimum length. For example, if we consider the primary items of the n queens problem in the natural order r_1, c_1, r_2, c_2, ..., rn, cn, queens tend to be placed at the top and left before we try to place them at the bottom and right. By contrast, if we use the “organ-pipe order” r⌊n/2⌋+1 ⁢, c⌊n/2⌋+1 ⁢, r⌊n/2⌋ ,c⌊n/2⌋ ,r⌊n/2⌋ +2,c⌊n/2⌋ +2,r⌊n/2⌋ −1,c⌊n/2⌋ −1, . . . , (r_1 or rn), (c_1 or cn), the queens are placed first in the center, where they prune the remaining possibilities more effectively. For example, the time needed to find all 14772512 ways to place 16 queens is 76 Gμ (gigamems) with the natural order, but only 40 Gμ with the organ-pipe order.

These running times can be compared with 9 Gμ, which we obtained using efficient bitwise operations with Algorithm 7.2.2W. Although that algorithm was specially tuned, the general-purpose dancing links technique runs only five times slower. Furthermore, the setup we’ve used here allows us to solve other problems that wouldn’t be anywhere near as easy with ordinary backtrack. For example, we can limit the solutions to those that contain queens on each of the 2 + 4l longest diagonals, simply by regarding as and bd as primary items instead of secondary, for |n +1 − s| ≤ l and |d| ≤ l. (There are 18048 such solutions when n = 16 and l = 4, found with organ-pipe order in 2.7 Gμ.)

We can also use secondary items to remove symmetry, so that most of the solutions are found only once instead of eight times (see exercises 22 and 23). The central idea is to use a pairwise ordering trick that works also in many other situations. Consider the following family of 2m options:

αj=‘a⁢ x0⁢ .⁢.⁢.⁢ xj−1’andβj=‘b⁢ xj’for 0≤j<m,(25)

where a and b are primary while x_0, x_1, ..., xm−1 are secondary. For example, when m = 4 the options are
α0=‘a’;β0=‘b⁢ x0’;α1=‘a⁢ x0’;β1=‘b⁢ x1’;α2=‘a⁢ x0⁢ x1’;β2=‘b⁢ x2’;α3=‘a⁢ x0⁢ x1⁢ x2’;β3=‘b⁢ x3’.

It’s not hard to see that there are exactly (m+12) ways to cover both a and b, namely to choose αj and βk with 0 ≤ j ≤ k < m. For if we choose αj, the secondary items x_0 through xj−1 knock out the options β_0, ..., βj−1.

This construction involves a total of (m+12) entries in the α options and 2m entries in the β options. But exercise 20 shows that it’s possible to achieve pairwise ordering with only O(m log m) entries in both α’s and β’s. For example, when m = 4 it produces the following elegant pattern:

α0=‘a’;β0=‘b⁢ y1⁢ y2’;α1=‘a⁢ y1’;β1=‘b⁢ y2’;α2=‘a⁢ ⁢ y2’;β2=‘b⁢ y3’;α3=‘a⁢  y3⁢ y2’;β3=‘b⁢ ’.(26)

Progress reports

Many of the applications of Algorithm X take a long time, especially when we’re using it to solve a tough problem that is breaking new ground. So we don’t want to just start it up and wait with our fingers crossed, hoping that it will finish soon. We really want to watch it in action and see how it’s doing. How many more hours will it probably run? Is it almost half done?

A simple amendment to step X2 will alleviate such worries. At the beginning of that step, as we enter a new node of the search tree, we can test whether the accumulated running time T has passed a certain threshold Θ, which is initially set to Δ. If T ≥ Θ, we print a progress report and set Θ ← Θ + Δ. (Thus if Δ = ∞, we get no reports; if Δ = 0, we see a report at each node.)

The author’s experimental program measures time in mems, so that he can obtain machine-independent results; and he usually takes Δ = 10 Gμ. His program has two main ways to show progress, namely a long form and a short form. The long form gives full details about the current state of the search, based on exercise 12. For example, here’s the first progress report that it displays when finding all solutions to the 16 queens problem as described above:

Click here to view code image

Current state (level 15):
r8 c3 ab ba (4 of 16)
c8 a8 bn r0 (1 of 13)
r7 cb ai bj (7 of 10)
r6 c4 aa bd (2 of 7)
. . .
3480159 solutions, 10000000071 mems, and max level 16 so far.

(The computer’s internal encoding for items is different from the conventions we have used; for example, ‘r8 c3 ab ba’ stands for what we called ‘r_9 c_4 a_13 b_5’. The first choice at level 0 was to cover item r8, meaning to put a queen into row 9. The fourth of 16 options for that item has placed it in column 4. Then at level 1 we tried the first of 13 ways to cover item c8, meaning to put a queen into column 9. And so on. At each level, the leftmost item shown for the option being tried is the one that was chosen in step X3 for branching.)

That’s the long form. The short form, which is the default, produces just one line for each state report:

Click here to view code image

10000000071mu: 3480159 sols, 4g 1d 7a 27 36 24 23 13 12 12 22 12 ... .19048
20000000111mu: 6604373 sols, 7g cd 6a 88 36 35 44 44 24 11 12 22 .43074
30000000052mu: 9487419 sols, bg cd 9a 68 37 35 24 13 12 12 .68205
40000000586mu: 12890124 sols, fg 6d aa 68 46 35 23 33 23 .90370
Altogether 14772512 solutions, 62296+45565990457 mems, 193032021 nodes.

Two-character codes are used to indicate the current position in the tree; for example, ‘4g’ means branch 4 of 16, then ‘1d’ means branch 1 of 13, etc. By watching these steadily increasing codes — it’s fun! — we can monitor the action.

Each line in the short form ends with an estimate of how much of the tree has been examined, assuming that the search tree structure is fairly consistent. For instance, ‘.19048’ means that we’re roughly 19% done. If we’re currently working at level l on choice cl of tl, this number is computed by the formula
c0−1t0+c1−1t0⁢ t1+⋅⋅⋅+c1−1t0⁢ t1.⁢ .⁢ .⁢ tl+12t0⁢ t1.⁢ .⁢ .tl.(27)

Sudoku

A “sudoku square” is a 9 × 9 array that has been divided into 3 × 3 boxes and filled with the digits {1, 2, 3, 4, 5, 6, 7, 8, 9} in such a way that

- every row contains each of the digits {1, 2, 3, 4, 5, 6, 7, 8, 9} exactly once;

- every column contains each of the digits {1, 2, 3, 4, 5, 6, 7, 8, 9} exactly once;

- every box contains each of the digits {1, 2, 3, 4, 5, 6, 7, 8, 9} exactly once.

(Since there are nine cells in each row, each column, and each box, the words ‘exactly once’ can be replaced by ‘at least once’ or ‘at most once’, anywhere in this definition.) Here, for example, are three highly symmetrical sudoku squares:

When the square has been only partially specified, the task of completing it — by filling in the blank cells — often turns out to be a fascinating challenge. Howard Garns used this idea as the basis for a series of puzzles that he called Number Place, first published in Dell Pencil Puzzles & Word Games #16 (May 1979), 6. The concept soon spread to Japan, where Nikoli Inc. gave it the name Su Doku (, “Unmarried Numbers”) in 1984; and eventually it went viral. By the beginning of 2005, major newspapers had begun to feature daily sudoku puzzles. Today, sudoku ranks among the most popular recreations of all time.

Every sudoku puzzle corresponds to an exact cover problem that has a particularly nice form. Consider, for example, the following three instances:

(The clues in (29a) match the first 32 digits of π; but the clues in (29b) and (29c) disagree with π after awhile.) For convenience, let’s number the rows, columns, and boxes from 0 to 8. Then every sudoku square S = (sij) corresponds naturally to the solution of a master exact cover problem whose 9 · 9 · 9 = 729 options are

‘pij⁢   rik⁢  cjk⁢  bxk’for 0≤i,j<9,1≤k≤9,and  x=3⌊i/3⌋+⌊j/3⌋,(30)

and whose 4 · 9 · 9 = 324 items are pij, rik, cjk, bxk. The reason is that option (30) is chosen with parameters (i, j, k) if and only if sij = k. Item pij must be covered by exactly one of the nine options that fill cell (i, j); item rik must be covered by exactly one of the nine options that put k in row i; ...; item bxk must be covered by exactly one of the nine options that put k in box x. Got it?

My own motive for writing on the subject is partly to justify
the appalling number of hours I have squandered solving Sudoku.

— BRIAN HAYES, in American Scientist (2006)

To find all sudoku squares that contain a given partial specification, we simply remove all of the items pij, rik, cjk, bxk that are already covered, as well as all of the options that involve any of those items. For example, (29a) leads to an exact cover problem with 4 · (81 − 32) = 196 items p_00, p_01, p_03, ..., r_02, r_04, r_05, ..., c_01, c_06, c_07, ..., b_07, b_08, b_09, ... ; it has 146 options, beginning with ‘p_00 r_07 c_07 b_07’ and ending with ‘p_88 r_86 c_86 b_86’. These options can be visualized by making a chart that shows every value that hasn’t been ruled out:

The active list for item p_00, say, has options for values {7, 8, 9}; the active list for item r_02 has options for columns {5, 6, 7}; the active list for item c_01 has options for rows {4, 5, 6, 7}; and so on. (Indeed, sudoku experts tend to have charts like this in mind, implicitly or explicitly, as they work.)

Aha! Look at the lonely ‘5’ in the middle! There’s only one option for p_44; so we can promote that ‘5’ to ‘5’. Hence we can also erase the other ‘5’s that appear in row 4, column 4, or box 4. This operation is called “forcing a naked single.”

And there’s another naked single in cell (8, 4). Promoting this one from ‘4’ to ‘4’ produces others in cells (7, 4) and (8, 2). Indeed, if the items pij have been placed first in step X1, Algorithm X will follow a merry path of forced moves that lead immediately to a complete solution of (29a), entirely via naked singles.

Of course sudoku puzzles aren’t always this easy. For example, (29b) has only 17 clues, not 32; that makes naked singles less likely. (Puzzle (29b) comes from Gordon Royle’s online collection of approximately 50,000 17-clue sudokus — all of which are essentially different, and all of which have a unique completion despite the paucity of clues. Royle’s collection appears to be nearly complete: Whenever a sudoku fanatic encounters a 17-clue puzzle nowadays, that puzzle almost invariably turns out to be equivalent to one in Royle’s list.)

A massive computer calculation, supervised by Gary McGuire and completed in 2012, has shown that every uniquely solvable sudoku puzzle must contain at least 17 clues. We will see in Section 7.2.3 that exactly 5,472,730,538 nonisomorphic sudoku squares exist. McGuire’s program examined each of them, and showed that comparatively few 16-clue subsets could possibly characterize it. About 16,000 subsets typically survived this initial screening; but they too were shown to fail. All this was determined in roughly 3.6 seconds per sudoku square, thanks to nontrivial and highly optimized bitwise algorithms. [See G. McGuire, B. Tugemann, and G. Civario, Experimental Mathematics 23 (2014), 190–217.]

The 17 clues of puzzle (29b) produce the following chart analogous to (31):

This one has 307 options remaining — more than twice as many as before. Also, as we might have guessed, it has no naked singles. But it still reveals forced moves, if we look more closely! For example, column 3 contains only one instance of ‘3’; we can promote it to ‘3’, and kill all of the other ‘3’s in row 2 and box 1. This operation is called “forcing a hidden single.”

Similarly, box 2 in (32) contains only one instance of ‘4’; and two other hidden singles are also present (see exercise 47). These forced moves cause other hidden singles to appear, and naked singles also arise soon. But after 16 forced promotions have been made, the low-hanging fruit is all gone:

Algorithm X readily deduces (33) from (32), because it sees naked singles and hidden singles whenever an item pij or rik or cjk or bxk has only one remaining option, and because its data structures change easily as the links dance. But when state (33) is reached, the algorithm resorts to two-way branching, in this case looking first at case ‘7’ of p_16, then backtracking later to consider case ‘8’.

A human sudoku expert would actually glance at (33) and notice that there’s a more intelligent way to proceed, because (33) contains a “naked pair”: Cells (4, 3) and (4, 8) both contain the same two choices; hence we’re allowed to delete ‘1’ and ‘7’ wherever they appear elsewhere in row 4; and this will produce a naked ‘4’ in column 1. Exercise 49 explores such higher-order deductions in detail.

Fancy logic that involves pairs and triples might well be preferable for earth-lings, but simple backtracking works just fine for machines. In fact, Algorithm X finds the solution to (29b) after exploring a search tree with just 89 nodes, the first 16 of which led it directly to (33). (It spends about 250 kilomems initializing the data structures in step X1, then 50 more kilomems to complete the task. Much more time would have been needed if it had tried to look for complicated patterns in step X3.) Here’s the solution that it discovers:

c33 b13 p23 r23 (1 of 1)
r13 c13 b03 p11 (1 of 1)
˙˙˙
c42 b72 p84 r82 (1 of 1)
p16 r18 c68 b28 (2 of 2)
b27 p18 r17 c87 (1 of 1)
˙˙˙
p85 r81 c51 b71 (1 of 1)

After it selects the correct value for column 6 of row 1, the rest is forced.

The dancing links method actually cruises to victory with amazing speed, on almost every known sudoku puzzle. Among several dozen typical specimens — seen by the author since 2005 in newspapers, magazines, books, and webpages worldwide, and subsequently presented to Algorithm X — roughly 70% were found to be solvable entirely by forced moves, based on naked or hidden singles, even though many of those puzzles had been rated ‘diabolical’ or ‘fiendish’ or ‘torturous’! Only 10% of them led to a search tree exceeding 100 nodes, and none of the trees had more than 282 nodes. (See, however, exercise 52.)

It’s interesting to consider what happens when the algorithm is weakened, so that its forced moves come only from naked singles, which are the easiest deductions for people to make. Suppose we classify the items rik, cjk, and bxk as secondary, leaving only pij as primary. (In other words, the specification will require at most one occurrence of each value k, in every row, every column, and every box, but it won’t explicitly insist that every k should be covered.) The search tree for puzzle (29b) then grows to a whopping 41,877 nodes.

Finally, what about puzzle (29c)? That one has only 16 clues, so we know that it cannot have a unique solution. But those 16 clues specify only seven of the nine digits; they give us no way to distinguish 7 from 8. Algorithm X deduces, with a 129-node search tree, that only two solutions exist. (Of course those two are essentially the same; they’re obtainable from each other by swapping 7 ↔ 8.)

Puzzlists have invented many intriguing variations on the traditional sudoku challenge, several of which are discussed in the exercises below. Among the best are “jigsaw sudoku puzzles” (also known as “geometric sudoku,” “polyomino sudoku,” “squiggly sudoku,” etc.), where the boxes have different shapes instead of simply being 3 × 3 subsquares. Consider, for example,

In puzzle (34a), which the author designed in 2017 with the help of Bob Harris, one can see for instance that there are only two places to put a ‘4’ in the top row, because of the ‘4’ in the next row. This puzzle is an exact cover problem just like (30), except that x is now a more complicated function of i and j. Similarly, Harris’s classic puzzle (34b) [Mathematical Wizardry for a Gardner (2009), 55– 57] asks us to put the letters {G, R, A, N, D, T, I, M, E} into each row, column, and irregularly shaped box. Again we use (30), but with the values of k running through letters instead of digits. [Hint: Cell (0, 2) must contain ‘A’, because column 2 needs an ‘A’ somewhere.] Puzzle (34c), The United States Jigsaw Sudoku, is a masterpiece designed and posted online by Thomas Snyder in 2006. It brilliantly uses boxes in the shapes of West Virginia, Kentucky, Wyoming, Alabama, Florida, Nevada, Tennessee, New York (including Long Island), and Virginia — and its clues are postal codes! (See exercise 59.)

Jigsaw sudoku was invented by J. Mark Thompson, who began to publish such puzzles in 1996 [GAMES World of Puzzles, #14 (July 1996), 51, 67] under the name Latin Squares. At that time he had not yet heard about sudoku; one of the advantages of his puzzles over normal sudoku was the fact that they can be of any size, not necessarily 9 × 9. Thompson’s first examples were 6 × 6.

The solutions to puzzles of this kind actually have an interesting prehistory: Walter Behrens, a pioneer in the applications of mathematics to agriculture, wrote an influential paper in 1956 that proposed using such patterns in empirical studies of crops that have been treated with various fertilizers [Zeitschrift für landwirtschaftliches Versuchs- und Untersuchungswesen 2 (1956), 176–193]. He presented dozens of designs, ranging from 4 × 4 to 10 × 10, including

Notice that Behrens’s (35b) is actually 9 × 7, so its rows don’t exhibit all 9 possibilities. He required only that no treatment number be repeated in any row or column. Notice also that his (35c) is actually a normal sudoku arrangement; this is the earliest known publication of what is now called a sudoku solution. Following a suggestion of F. Ragaller, Behrens called these designs “gerechte” (“equitable”) latin squares or latin rectangles, because they assign neighborhood groupings to tracts of land that have been subjected to all n treatments.

All of his designs were partitions of rectangles into connected regions, each with n square cells. We’ll see next that that idea actually turns out to have its own distinguished history of fascinating combinatorial patterns and recreations.

Polyominoes

A rookwise-connected region of n square cells is often called an n-omino, following a suggestion by S. W. Golomb [AMM 61 (1954), 675–682]. When n = 1, 2, 3, ..., Golomb’s definition gives us monominoes, dominoes, trominoes, tetrominoes, pentominoes, hexominoes, and so on. In general, when n is unspecified, Golomb called such regions polyominoes.

We’ve already encountered small polyominoes, together with their relation to exact covering, in 7.1.4–(130). It’s clear that a domino has only one possible shape. But there are two distinct species of trominoes, one of which is “straight” (1 × 3) and the other is “bent,” occupying three cells of a 2 × 2 square. Similarly, the tetrominoes can be classified into five distinct types. (Can you draw all five, before looking at exercise 274? Tetris® players will have no trouble doing this.)

The most piquant polyominoes, however, are almost certainly the pentominoes, of which there are twelve. These twelve shapes have become the personal friends of millions of people, because they can be put together in so many elegant ways. Sets of pentominoes, made from finely crafted hardwoods or from brilliantly colored plastic, are readily available at reasonable cost. Every home really ought to have at least one such set — even though “virtual” pentominoes can easily be manipulated in computer apps — because there’s no substitute for the strangely fascinating tactile experience of arranging these delightful physical objects by hand. Furthermore, we’ll see that pentominoes have much to teach us about combinatorial computing.

If mounted on cardboard, [these pieces]
will form a source of perpetual amusement in the home.

— HENRY E. DUDENEY, The Canterbury Puzzles (1907)

Which English nouns ending in -o pluralize with -s and which with -es?
If the word is still felt as somewhat alien, it takes -s,
while if it has been fully naturalized into English, it takes -es.
Thus, echoes, potatoes, tomatoes, dingoes, embargoes, etc.,
whereas Italian musical terms are altos, bassos, cantos, pianos, solos, etc.,
and there are Spanish words like tangos, armadillos, etc.
I once held a trademark on ‘Pentomino(-es)’, but I now prefer
to let these words be my contribution to the language as public domain.

— SOLOMON W. GOLOMB, letter to Donald Knuth (16 February 1994)

One of the first things we might try to do with twelve pieces of 5 cells each is to pack them into a rectangular box, either 6 × 10 or 5 × 12 or 4 × 15 or 3 × 20. The first three tasks are fairly easy; but a 3 × 20 box presents more of a challenge. Golomb posed this question in his article of 1954, without providing any answer. At that time he was unaware that Frans Hansson had already given a solution many years earlier, in an obscure publication called The Problemist: Fairy Chess Supplement 2, 12 and 13 (June and August, 1935), problem 1844:

Hansson had in fact observed that the bracketed pieces “may also be rotated through two right angles, to give the only other possible solution.”

This problem, and many others of a similar kind, can be formulated nicely in terms of exact covering. But before we do this, we need names for the individual pentomino shapes. Everybody agrees that seven of the pentominoes should be named after seven consecutive letters of the alphabet:

But two different systems of nomenclature have been proposed for the other five:

where Golomb liked to think of the word ‘Filipino’ while Conway preferred to map the twelve pentominoes onto twelve consecutive letters from O to Z. Conway’s scheme tends to work better in computer programs, so we’ll use it here.

The task of 3 × 20 pentomino packing is to arrange pentominoes in such a way that every piece name {O, P, ..., Z} is covered exactly once, and so is every cell ij for 0 ≤ i < 3 and 0 ≤ j < 20. Thus there are 12 + 3 · 20 = 72 items; and there’s an option for each way to place an individual pentomino, namely

‘0⁢ 00⁢  01⁢ 02⁢ 03⁢ 04’.⁢.⁢.‘0⁢  2f 2g 2h 2i 2j’‘p 00 01 02 10 11’.⁢.⁢.‘z 0j 1h 1i 1j 2h’(37)

if we extend hexadecimal notation so that the “digits” (a, b, ..., j) represent (10, 11, ..., 19). In this list, pieces (O, P, ..., Z) contribute respectively (48, 220, 136, 144, 136, 72, 110, 72, 72, 18, 136, 72) options, making 1236 altogether. Exercise 266 explains how to generate all of the options for problems like this.

When Algorithm X is applied to (37), it finds eight solutions, because each of Hansson’s arrangements is obtained with horizontal and/or vertical reflection. We can remove that symmetry by insisting that the V pentomino must appear in its ‘Γ-like’ orientation, as it does in (36), namely by removing all but 18 of its 72 options. (Do you see why? Think about it.) Without that simplification, a 32,644-node search tree finds 8 solutions in 146 megamems; with it, a 21,805-node search tree finds 2 solutions in 103 megamems.

A closer look shows that we can actually do much better. For example, one of the Γ-like options for V is ‘V 09 0a 0b 19 29’, representing

but this placement could never be used, because it asks us to pack pentominoes into the 27 cells at V’s left. Many of the options for other pieces are similarly unusable, because (like (38)) they isolate a region whose area isn’t a multiple of 5.

In fact, if we remove all such options, only 728 of the original 1236 potential placements remain; they include respectively (48, 156, 132, 28, 128, 16, 44, 16, 12, 4, 128, 16) placements of (O, P, ..., Z). That gives us 716 options, when we remove also the 12 surviving placements for V that make it non-‘Γ’. When Algorithm X is applied to this reduced set, the search tree for finding all solutions goes down to 1243 nodes, and the running time is only 4.5 megamems.

(There’s also a slightly better way to remove the symmetry: Instead of insisting that piece V looks like ‘Γ’ we can insist that piece X lies in the left half, and that piece Z hasn’t been “flipped over.” This implies that there are (16, 2, 8) potential placements for (V, X, Z), instead of (4, 4, 16). The resulting search tree has just 1128 nodes, and the running time is 4.0 Mμ.)

Notice that we could have begun with a weaker formulation of this problem: We could merely have asked for pentomino arrangements that use each piece at most once, while covering each cell ij exactly once. That would be essentially the same as saying that the piece names {O, P, ..., Z} are secondary items instead of primary. Then the original set of 1236 options in (37) would have led to a search tree with 61,851 nodes, and a runtime of 291 Mμ. Dually, we could have kept the piece names primary but made the cell names secondary; that would have yielded a 1,086,521,921-node tree, with a runtime of 2.94 Tμ! These statistics are curiously reversed, however, with respect to the reduced set of 716 options obtained by discarding cases like (38): Then piece names secondary yields 19306 nodes (68 Mμ); cell names secondary yields 11656 nodes (37 Mμ).

In the early days of computing, pentomino problems served as useful benchmarks for combinatorial calculations. Programmers didn’t have the luxury of large random-access memory until much later; therefore techniques such as dancing links, in which more than a thousand options are explicitly listed and manipulated, were unthinkable at the time. Instead, the options for each piece were implicitly generated on-the-fly as needed, and there was no incentive to use fancy heuristics while backtracking. Each branch of the search was essentially based on the available ways to cover the first cell ij that hadn’t yet been occupied.

We can simulate the behavior of those historic methods by running Algorithm X without the MRV heuristic and simply setting i ← RLINK(0) in step X3. An interesting phenomenon now arises: If the cells ij are considered in their natural order — first 00, then 01, ..., then 0j, then 10, ..., finally 2j — the search tree has 1.5 billion nodes. (There are 29 ways to cover 00; if we choose ‘00 01 02 03 04 0’ there are 49 ways to cover 05; and so on.) But if we consider the 20 × 3 problem instead of 3 × 20, so that the cells ij for 0 ≤ i < 20 and 0 ≤ j < 3 are processed in order 00, 01, 02, 10, ..., 2j, the search tree has just 71191 nodes, and all eight solutions are found very quickly. (This speedup is mostly due to having a better “focus,” which we’ll discuss later.) Again we see that a small change in problem setup can have enormous ramifications.

The best of these early programs were highly tuned, written in assembly language with ingenious uses of macro instructions. Memwise, they were therefore superior to Algorithm X on smallish problems. [See J. G. Fletcher, CACM 8,10 (October 1965), cover and 621–623; N. G. de Bruijn, FGbook pages 465–466.] But the MRV heuristic eventually wins, as problems get larger.

Exercises 268–323 discuss some of the many intriguing and instructive problems that arise when we explore the patterns that can be made with pentominoes and similar families of planar shapes. Several of these problems are indeed large — beyond the capabilities of today’s machines.

Polycubes

And if you think two-dimensional shapes are fun, you’ll probably enjoy three dimensions even more! A polycube is a solid object formed by taking one or more 1 × 1 × 1 cubes and joining them face-to-face. We call them mono-cubes, dicubes, tricubes, tetracubes, pentacubes, etc.; but we don’t call them “n-cubes” when they’re made from n unit cubies, because mathematicians have reserved that term for n-dimensional objects.

A new situation arises when n = 4. In two dimensions we found it natural to regard the tetromino ‘’ as identical to its mirror image ‘’, because we could simply flip it over. But the tetracube is noticeably different from its mirror reflection , because we can’t change one into the other without going into the fourth dimension. Polycubes that differ from their mirror images are called chiral, a word coined by Lord Kelvin in 1893 when he studied chiral molecules.

The simplest polycubes are cuboids — also called “rectangular parallelepipeds” by people who like long names — which are like bricks of size l × m × n. But things get particularly interesting when we consider noncuboidal shapes. Piet Hein noticed in 1933 that the seven smallest shapes of that kind, namely

can be put together to form a 3 × 3 × 3 cube, and he liked the pieces so much that he called them Soma. Notice that the first four pieces are essentially planar, while the other three are inherently three-dimensional. The twists are chiral.

Martin Gardner wrote about the joys of Soma in Scientific American 199,3 (September 1958), 182–188, and it soon became wildly popular: More than two million SOMA® cubes were sold in America alone, after Parker Brothers began to market a well-made set together with an instruction booklet written by Hein.

A minimum number of blocks of simple form are employed. . . .
Experiments and calculations have shown that from the set of seven blocks
it is possible to construct approximately the same number of geometrical
figures as could be constructed from twenty-seven separate cubes.

— PIET HEIN, United Kingdom Patent Specification 420,349 (1934)

The task of packing these seven pieces into a cube is easy to formulate as an exact cover problem, just as we did when packing pentominoes. But this time we have 24 3D-rotations of the pieces to consider, instead of 8 2D-rotations and/or 3D-reflections; so exercise 324 is used instead of exercise 266 to generate the options of the problem. It turns out that there are 688 options, involving 34 items that we can call 1, 2, ..., 7, 000, 001, ..., 222. For example, the first option

‘1⁢   000⁢   001⁢  010’(40)

characterizes one of the 144 potential ways to place the “bent” piece 1.

Algorithm X needs just 407 megamems to find all 11,520 solutions to this problem. Furthermore, we can save most of that time by taking advantage of symmetry: Every solution can be rotated into a unique “canonical” solution in which the “ell” piece 2 has not been rotated; hence we can restrict that piece to only six placements, namely (000, 010, 020, 100), (001, 011, 021, 101), ... ,(102, 112, 122, 202) — all shifts of each other. This restriction removes 138=2324 · 144 options, and the algorithm now finds the 480 canonical solutions in just 20 megamems. (These canonical solutions form 240 mirror-image pairs.)

Factoring an exact cover problem

In fact, we can simplify the Soma cube problem much further, so that all of its solutions can actually be found by hand in a reasonable time, by factoring the problem in a clever way.

Let’s observe first that any solution to an exact cover problem automatically solves infinitely many other problems. Going back to our original formulation in terms of an m × n matrix A = (aij), the task is to find all sets of rows whose sum is 1 in every column, namely to find all binary vectors x_1 ... xm such that ∑i=1mxiaij=1⁢  for 1 ≤ j ≤ n. Therefore if we set bi = α_1ai1 + ··· + αnain for 1 ≤ i ≤ m, where (α_1, ..., αn) is any n-tuple of coefficients, the vectors x_1 ... xm will also satisfy ∑i=1mxibi=α1+⋅⋅⋅+αn⁢ . By choosing the α’s intelligently we may be able to learn a lot about the possibilities for x_1 ... xm.

For example, consider again the 6 × 7 matrix A in (5), and let α_1 = ··· = α_7 = 1. The sum of the entries in each row of A is either 2 or 3; thus we’re supposed to cover 7 things, by burying either 2 or 3 at a time. Without knowing anything more about the detailed structure of A, we can conclude immediately that there’s only one way to obtain a total of 7, namely by selecting 2 + 2 + 3! Furthermore, only rows 1 and 5 have 2 as their sum; we must choose them.

Now here’s a more interesting challenge: “Cover the 64 cells of a chessboard with 21 straight trominoes and one monomino.” This problem corresponds to a big matrix that has 96 + 64 rows and 1 + 64 columns,

(M000102030405060710111213141516172021222324747576770111000000000000000000...00000100000001000000010000...00000011110000000000000000...00000010000000100000001000...0000.......................0000000000000000000000...01111100000000000000000000...00001010000000000000000000...0000.......................1000000000000000000000...0001),(41)

where the first 96 rows specify all possible ways to place a tromino and the other 64 rows specify the possibilities for the monomino. Column ij represents cell (i, j); column ‘M’ means “monomino.”

The three cells (i, j) covered by a straight tromino always lead to distinct values of (i − j) mod 3. Therefore, if we add up the 22 columns of (41) for which (i − j) mod 3 = 0, we get 1 in each of the first 96 rows, and 0 or 1 in the other 64 rows. We’re supposed to get a total of 22 in the chosen rows; hence the monomino has to go into a cell (i, j) with i ≡ j (modulo 3).

A similar argument, using i + j instead of i − j, shows that the monomino must also go into a cell with i + j ≡ 1 (modulo 3). Therefore i ≡ j ≡ 2. We’ve proved that there are only four possibilities for (i, j), namely (2, 2), (2, 5), (5, 2), (5, 5). [Golomb made this observation in his 1954 paper that introduced polyominoes, after “coloring” the cells of a chessboard with three colors. The general notion of factoring includes all such coloring arguments as special cases.]

Our proof that (38) is an impossible pentomino placement can also be regarded as an instance of factorization. The residual problem, if (38) is chosen, has a total of either 0 or 5 in the first 27 columns of each remaining row of the associated matrix. Therefore we can’t achieve a total of 27 from those rows.

Consider now a three-dimensional problem [J. Slothouber and W. Graatsma, Cubics (1970), 108–109]: Can six 1 × 2 × 2 cuboids be packed into a 3 × 3 × 3 box? This is the problem of choosing six rows of the 36 × 27 matrix

(000001002010011012020021022100101102110111112120121122200201202210211212220221222110110000000000000000000000110000000110000000000000000100100000100100000000000000011011000000000000000000000011000000011000000000000000010010000010010000000000000001001000001001000000000000.......................000000000000000000000011011),(42)

in such a way that all of the column sums are ≤ 1.

The 27 cubies (i, j, k) of a 3 × 3 × 3 cube fall into four classes, depending on how many of its coordinates have the middle value 1:

A vertex cubie has no 1s.((30)23=8⁢  cases.)An edge cubie has one 1.((31)22=12⁢  cases.)A face cubie has two 1s.((32)21=6⁢  cases.)(43)A central cubie has three 1s.((33)20=1⁢  cases.)

Every symmetry of the cube preserves these classes.

Imagine placing four new columns v, e, f, c at the right of (42), representing the number of vertex, edge, face, and central cubies of a placement. Then 24 of the rows will have (v, e, f, c) = (1, 2, 1, 0), and the other 12 rows will have (υ, e, f, c) = (0, 1, 2, 1). If we choose a rows of the first kind and b rows of the second kind, this factorization tells us that we must have
a≥0,b≥0, a+b=6,a≤8,2a+b≤12,a+2b≤6,b≤1.(44)

That’s more than enough to prove that b = 0 and a = 6, and thus to find the essentially unique way to pack those six cuboids.

(We could paraphrase this argument as follows, making it more impressive by concealing the low-level algebra that inspired it: “Each 1 × 2 × 2 cuboid occupies at least one face cubie. So each of them must be placed on a different face.”)

With these examples in mind, we’re ready now to apply factorization to the Soma cube. The possible (v, e, f, c) values for pieces 1 through 7 in (39) are:

Piece 1:(0,1,1,1),(0,0,2,1),(0,1,2,0),(0,2,1,0),(1,1,1,0),(1,2,0,0).Piece 2:(0,1,2,1),(0,2,2,0),(1,2,1,0),(2,2,0,0).Piece 3:(0,0,3,1),(0,2,1,1),(0,3,1,0),(2,1,1,0).Piece 4:(0,1,2,1),(1,2,1,0).Piece 5:(0,1,2,1),(0,2,2,0),(1,1,1,1),(1,2,1,0).Piece 6:(0,1,2,1),(0,2,2,0),(1,1,1,1),(1,2,1,0).Piece 7:(0,2,1,1),(0,0,3,1),(1,1,2,0),(1,3,0,0).(45)

(This is actually much more information than we need, but it doesn’t hurt.)

Looking only at the totals for v, we see that we must have

(0 or 1) + (0, 1, or 2) + (0 or 2) + (0 or 1) + (0 or 1) + (0 or 1) + (0 or 1) = 8;
and the only way to achieve this is via

(0 or 1) + (1 or 2) + 2 + (0 or 1) + (0 or 1) + (0 or 1) + (0 or 1) = 8,
thus eliminating several options for pieces 2 and 3. More precisely, piece 2 must touch at least one vertex; piece 3 must be placed along an edge.

Looking next at the totals for v + f, which are the “black” cubies if we color them alternately black and white with black in the corners, we must also have

(1 or 2) + 2 + 3 + 2 + 2 + 2 + (1 or 3) = 14;
and the only way to achieve this is with two from piece 1 and one from piece 7: Piece 1 must occupy two black cubies, and piece 7 must occupy just one.

We have therefore eliminated 200 of the 688 options in the list that begins with (40). And we also know that exactly five of the pieces 1, 2, 4, 5, 6, 7 occupy as many of the corner vertices as they individually can. This extra information can be encoded by introducing 13 new primary items

*,1+,1−,2+,2−,4+,4−,5+,5−,6+,6−,7+,7−(46)

and six new options

‘*1+2−4−5−6−7−’‘*1−2+4−5−6−7−’‘*1−2−4+5−6−7−’‘*1−2−4−5+6−7−’‘*1−2−4−5−6+7−’‘*1−2−4−5−6−7+’(47)

and by appending p+ or p- to each of piece p’s options that do or don’t touch the most corners. For example, this new set of 6 + 488 options for the Soma cube problem includes the following typical ways to place various pieces:

‘1 000 001 011 1+’
‘1 001 011 101 1-’
‘2 000 001 002 010 2+’
‘2 000 001 011 021 2-’
‘3 000 001 002 011’
‘4 000 001 011 012 4+’
‘4 001 011 111 121 4-’
‘5 000 001 010 110 5+’
‘5 001 010 011 101 5-’
‘6 000 001 010 101 6+’
‘6 001 010 011 110 6-’
‘7 000 001 010 100 7+’
‘7 001 010 011 111 7-’

As before, Algorithm X finds 11,520 solutions; but now it needs only 108 megamems to do so. Each of the new options is used in at least 21 of the solutions, hence we’ve removed all of the “fat” in the original set. [This instructive analysis of Soma is due to M. J. T. Guy, R. K. Guy, and J. H. Conway in 1961. See Berlekamp, Conway, and Guy, Winning Ways, second edition (2004), 845–847.]

To reduce the number of solutions, using symmetry, we can force piece 3 to occupy the cells {000, 001, 002, 011} (thus saving a factor of 24), and we can remove all options for piece 7 that use a cell ijk with k = 2 (saving an additional factor of 2). From the remaining 455 options, Algorithm X needs just 2 megamems to generate all 240 of the essentially distinct solutions.

The seven Soma pieces are amazingly versatile, and so are the other poly-cubes of small sizes. Exercises 324–350 explore some of their remarkable properties, together with historical references.

Color-controlled covering

Take a break! Before reading any further, please spend a minute or two solving the “word search” puzzle in Fig. 71. Comparatively mindless puzzles like this one provide a low-stress way to sharpen your word-recognition skills. It can be solved easily — for instance, by making eight passes over the array — and the solution can be found in Fig. 72 on the next page.

Fig. 71. Find the mathematicians*: Put ovals around the following names where they appear in the 15 × 15 array shown here, reading either forward or backward or upward or downward, or diagonally in any direction. After you’ve finished, the leftover letters will form a hidden message. (The solution appears on the next page.)

* The journal Acta Mathematica celebrated its 21st birthday by publishing a special Table Générale des Tomes 1–35, edited by Marcel Riesz (Uppsala: 1913), 179 pp. It contained a complete list of all papers published so far in that journal, together with portraits and brief biographies of all the authors. The 27 mathematicians mentioned in Fig. 71 are those who were subsequently mentioned in Volumes 1, 2, or 3 of The Art of Computer Programming — except for people like MITTAG-LEFFLER or POINCARÉ, whose names contain special characters.

Click here to view code image

ABEL HENSEL MELLIN
BERTRAND HERMITE MINKOWSKI
BOREL HILBERT NETTO
CANTOR HURWITZ PERRON
CATALAN JENSEN RUNGE
FROBENIUS KIRCHHOFF STERN
GLAISHER KNOPP STIELTJES
GRAM LANDAU SYLVESTER
HADAMARD MARKOFF WEIERSTRASS

Our goal in this section is not to discuss how to solve such puzzles; instead, we shall consider how to create them. It’s by no means easy to pack those 27 names into the box in such a way that their 184 characters occupy only 135 cells, with eight directions well mixed. How can that be done with reasonable efficiency?

For this purpose we shall extend the idea of exact covering by introducing color codes. Let’s imagine that each cell ij of the array is to be “colored” with one of the letters {A, ..., Z}. Then the task of creating such a puzzle is essentially to choose from among a vast set of options

‘ABEL 00 : A 01: B 02: E 03: L’‘ABEL 00 : A 10: B 20: E 30: L’‘ABEL 00 : A 11: B 22: E 33: L’‘ABEL 00 : L 01: E 02: B 03: A’    .⁢      .⁢     .⁢     .⁢    .‘WEIERSTASS e4: S e5: S e6: A e7: R e8: T e9: S ea: R eb: E ec: I ed: E ee: W’(48)

in such a way that the following conditions are satisfied:

1. Exactly one option must be chosen for each of the 27 mathematicians’ names.

2. The chosen options must give consistent colors to each of the 15×15 cells ij.

Fig. 72. Solution to the puzzle of the hidden mathematicians (Fig. 71). Notice that the central letter R actually participates in six different names:

BERTRAND
GLAISHER
HERMITE
HILBERT
KIRCHHOFF
WEIERSTRASS

The T to its left participates in five.

Here’s what the leftover letters say:

These authors of early papers in Acta Mathematica were cited years later in The Art of Computer Programming.

There also are informal constraints: It’s desirable to have many shared letters between names, and to intermix the various directions, so that the puzzle has plenty of variety and perhaps a few surprises. But conditions (i) and (ii) are the important criteria for a computer to consider; the auxiliary informalities are best handled interactively, with human guidance.

Notice that the color constraints (ii) are significantly different from the name constraints (i). Several distinct options are allowed to specify the color of the same cell, as long as those specifications don’t conflict with each other.

Let us therefore define a new problem, exact covering with colors, or XCC for short. As before, we’re given a set of items, of which N_1 are primary and N − N_1 are secondary. We’re also given a family of M options, each of which includes at least one primary item. The new rule is that a color is assigned to the secondary items of each option. The new task is to find all choices of options such that

1. every primary item occurs exactly once; and

2. every secondary item has been assigned at most one color.

The primary items are required; the secondary items are elective.

Color assignments are denoted by a colon; for example, ‘00:A’ in (48) means that color A is assigned to the secondary item 00. When a secondary item of an option is not followed by a colon, it is implicitly assigned a unique color, which doesn’t match the color of any other option. Therefore the ordinary exact cover problems that we’ve been studying so far, in which secondary items don’t explicitly receive colors but cannot be chosen in more than one option, are just special cases of the XCC problem, even though nothing about color was mentioned.

A tremendous variety of combinatorial problems can be expressed readily in the XCC framework. And there’s good news: The dancing links technique works beautifully with such problems! Indeed, we will see that this considerably more general problem can be solved with only a few small extensions to Algorithm X.

The nodes of Algorithm X have just three fields: TOP, ULINK, and DLINK. We now add a fourth field, COLOR; this field is set to the positive value c when the node represents an item that has explicitly been assigned color c. Consider, for example, the following toy problem with three primary items {p, q, r} and two secondary items {x, y}, where the options are

‘pq  x  y : A’⁢ ;‘pr  x : A  y’⁢ ;‘px : B’⁢  ;‘qx : A’⁢  ;‘ry : B’⁢  .(49)

Table 2 shows how it would be represented in memory, extending the conventions of Table 1. Notice that COLOR = 0 when no color has been specified. The COLOR fields of the header nodes (nodes 1–5 in this example) need not be initialized because they’re never examined except in printouts (see the answer to exercise 12). The COLOR fields of the spacer nodes (nodes 6, 11, 16, 19, 22, 25) are unimportant, except that they must be nonnegative.

Table 2
THE INITIAL CONTENTS OF MEMORY CORRESPONDING TO (49)

i:

0

1

2

3

4

5

6

NAME(i):

—

p

q

r

x

y

—

LLINK(i):

3

0

1

2

6

4

5

RLINK(i):

1

2

3

0

5

6

4

x:

0

1

2

3

4

5

6

LEN(x), TOP(x):

—

3

2

2

4

3

0

ULINK(x):

—

17

20

23

21

24

—

DLINK(x):

—

7

8

13

9

10

10

COLOR(x):

—

—

—

—

—

—

0

x:

7

8

9

10

11

12

13

TOP(x):

1

2

4

5

−1

1

3

ULINK(x):

1

2

4

5

7

7

3

DLINK(x):

12

20

14

15

15

17

23

COLOR(x):

0

0

0

A

0

0

0

x:

14

15

16

17

18

19

20

TOP(x):

4

5

−2

1

4

−3

2

ULINK(x):

9

10

12

12

14

17

8

DLINK(x):

18

24

18

1

21

21

2

COLOR(x):

A

0

0

0

B

0

0

x:

21

22

23

24

25

TOP(x):

4

−4

3

5

−5

ULINK(x):

18

20

13

15

23

DLINK(x):

4

24

3

5

—

COLOR(x):

A

0

0

B

0

It’s easy to see how these COLOR fields can be used to get the desired effect: When an option is chosen, we “purify” any secondary items that it names, by effectively removing all options that have conflicting colors. One slightly subtle point arises, because we don’t want to waste time purifying a list that has already been culled. The trick is to set COLOR(x) ← −1 in every node x that’s already known to have the correct color, except in nodes that have already been hidden.

Thus we want to upgrade the original operations cover (i) and hide (p) in (12) and (13), as well as their counterparts uncover (i) and unhide (p) in (14) and (15), in order to incorporate color controls. The changes are simple:

cover'(i) is like cover(i), but it calls hide'(p)   insted of hide(p);(50)

hide'(p) is like hide(p), but it ignores node q⁢  when COLOR (q) < 0;(51)

uncover'(i) is like uncover(i), but it calls unhide'(p) instead of unhide(p);(52)

unhide'(p) is like unhide(p), but it ignores node q⁢ when COLOR (q) < 0.(53)

Our colorful algorithm also introduces two new operations and their inverses:

commit(p,j) ={if COLOR (p) = 0, cover' (j)if COLOR (p) > 0, purify (p)(54)

purify(p)={Set c⁢ ←COLOR(p), i⁢ ←TOP (p), COLOR (i) ← c, q⁢ ←DLINK(i).While q⁢ ≠i,⁢do the following and set q⁢ ←DLINK (q):             if COLOR (q) = c, set COLOR(q) ← −1;             otherwise hide'(q).(55)

uncommit (p,j) = {If COLOR(p) = 0, uncover'(j);if COLOR (p) > 0, unpurify (p).(56)

unpurify (p) = {Set c⁢←COLOR(p), i⁢←TOP (p), q⁢←ULINK (i).While  q≠i, do the following and set q⁢ ←ULINK (q) :               if COLOR (q) < 0, set COLOR (q) ←c;              otherwise unhide'(q).(57)

Otherwise Algorithm C is almost word-for-word identical to Algorithm X.

Algorithm C (Exact covering with colors). This algorithm visits all solutions to a given XCC problem, using the same conventions as Algorithm X.

C1. [Initialize.] Set the problem up in memory, as in Table 2. (See exercise 8.) Also set N to the number of items, Z to the last spacer address, and l ← 0.

C2. [Enter level l.] If RLINK(0) = 0 (hence all items have been covered), visit the solution that is specified by x_0x_1 ... xl−1 and go to C8. (See exercise 13.)

C3. [Choose i.] At this point the items i_1, ..., it still need to be covered, where i_1 = RLINK(0), ij+1 = RLINK(ij), RLINK(it) = 0. Choose one of them, and call it i. (The MRV heuristic of exercise 9 often works well in practice.)

C4. [Cover i.] Cover item i using (50), and set xl ← DLINK(i).

C5. [Try xl.] If xl = i, go to C7 (we’ve tried all options for i). Otherwise set p ← xl + 1, and do the following while p ≠ xl: Set j ← TOP(p); if j ≤ 0, set p ← ULINK(p); otherwise commit(p, j) and set p ← p + 1. (This commits the items ≠ i in the option that contains xl.) Set l ← l +1 and return to C2.

C6. [Try again.] Set p ← xl − 1, and do the following while p ≠ xl: Set j ← TOP(p); if j ≤ 0, set p ← DLINK(p); otherwise uncommit(p, j) and set p ← p − 1. (This uncommits the items ≠ i in the option that contains xl, using the reverse order.) Set i ← TOP(xl), xl ← DLINK(xl), and return to C5.

C7. [Backtrack.] Uncover item i using (52).

C8. [Leave level l.] Terminate if l = 0. Otherwise set l ← l − 1 and go to C6.

Algorithm C applies directly to several problems that we’ve already discussed in previous sections. For example, it readily generates word rectangles, as well as intriguing patterns of words that have more intricate structure (see exercises 87–93). We can use it to find all de Bruijn cycles, and their two-dimensional counterparts (see exercises 94–97). The extra generality of exact covering options also invites us to impose additional constraints for special applications. Furthermore, Algorithm C facilitates experiments with the tetrad tiles that we studied in Section 2.3.4.3 (see exercises 120 and 121).

The great combinatorialist P. A. MacMahon introduced several families of colorful geometric patterns that continued to fascinate him throughout his life. For example, in U.K. Patent 3927 of 1892, written with J. R. J. Jocelyn, he considered the 24 different triangles that can be made with four colors on their edges,

and showed two ways in which they could be arranged to form a hexagon with matching colors at adjacent edges and with solid colors on the outer boundary:

(Notice that chiral pairs, like and in (58), are considered to be distinct; MacMahon’s tiles can be rotated, but they can’t be “flipped over.”)

Four suitable colours are black, white, red, and blue,
as they are readily distinguishable at night.

— P. A. MACMAHON, New Mathematical Pastimes (1921)

Let’s assume that the boundary is supposed to be all white, as in pattern (59b). There are millions of ways to satisfy this condition; but every really distinct solution is counted 72 times, because the hexagon has 12 symmetries under rotation and reflection, and because the three nonwhite colors can be permuted in 3! = 6 ways. We can remove the hexagon symmetries by fixing the position of the all-white triangle (see exercise 119). And the color symmetries can be removed by using an interesting extension of Algorithm C, which reduces the number of solutions by a factor of d! when the options are symmetrical with respect to d colors (see exercise 122). In this way all of the solutions — can you guess how many? — can actually be found with only 5.2 Gμ of computation (see exercise 126).

MacMahon went on to study many other matching problems with these triangles, as well as with similar sets of tiles that are based on squares, hexagons, and other shapes. He also considered three-dimensional arrangements of colored cubes, which are supposed to match where they touch. Exercises 127–148 are devoted to some of the captivating questions that have arisen from this work.

Introducing multiplicity

We’ve now seen from numerous examples that Algorithm C — which extends Algorithm X and solves arbitrary XCC problems — is enormously versatile. In fact, there’s a sense in which every constraint satisfaction problem is a special case of an XCC problem (see exercise 100).

But we can extend Algorithm C even further, again without substantial changes, so that it goes well beyond the original notion of exact covering. For example, let’s consider Robert Wainwright’s “partridge puzzle” (1981), which was inspired by the well-known fact that the sum of the first n cubes is a perfect square:

13+23+⋅⋅⋅+n3=N2,where   N⁢= 1+2+⋅⋅⋅+n.(60)

Wainwright wondered if this relation could be verified geometrically, by taking one square of size 1 × 1, two squares of size 2 × 2, ..., n squares of size n × n, and packing them all into a big square of size N ×N. (We know from exercise 1.2.1–8 that 4k squares of each size k × k can be packed into a 2N × 2N square. But Wainwright hoped for a more direct corroboration of (60).) He proved the task impossible for 2 ≤ n ≤ 5, but found a perfect packing when n = 12; so he thought of the 12 days of Christmas, and named his puzzle accordingly (see exercise 154).

This partridge puzzle is easily expressed in terms of options that involve n items #k for 1 ≤ k ≤ n, as well as N^2 items ij for 0 ≤ i, j < N. By analogy with what we did with pentominoes in (37), the options are

‘#k⁢ij⁢  i(j=1).⁢  .⁢ ⁢ ⁢.⁢   i(j+k−1)  (i+1)j(i+1)  (j+1) .⁢ ⁢.⁢ ⁢.⁢  (i+k−1)(j+k−1)’(61)

for 1 ≤ k ≤ n and 0 ≤ i, j ≤ N − k. (Exactly (N + 1 − k)2 options involve #k, and each of them names 1 + k^2 items.) For example, the options when n = 2 are

‘#1 00’, ‘#1 01’, ‘#1 02’, ‘#1 10’, ‘#1 11’, ‘#1 12’, ‘#1 20’, ‘#1 21’, ‘#1 22’,

‘#2 00 01 10 11’, ‘#2 01 02 11 12’, ‘#2 10 11 20 21’, ‘#2 11 12 21 22’.

As before, we want to cover each of the N^2 cells ij exactly once. But there’s a difference: We now want to cover primary item #k exactly k times, not just once.

That’s a rather big difference. But in Algorithm M below, we’ll see that the dancing links approach can handle it. For example, that algorithm can show that the partridge puzzle has no perfect packings for n = 6 or n = 7; but it finds thousands of surprising solutions when n = 8, such as

When we first defined exact cover problems, near the beginning of this section, we considered M × N matrices of 0s and 1s, such as (5). In matrix terms, the task was to find all subsets of the rows whose sum is 11 ... 1. Algorithm M is going to do much more: It will find all subsets of rows whose sum is v_1v_2 ... vN , where v_1v_2 ... vN is any desired vector of multiplicities.

In fact, Algorithm M will go further yet, by allowing intervals [uj .. vj] to be prescribed for each multiplicity. It will actually solve the general MCC problem, “multiple covering with colors,” which is defined as follows: There are N items, of which N_1 are primary and N − N_1 are secondary. Each primary item j for 1 ≤ j ≤ N_1 is assigned an interval [uj .. vj] of permissible values, where 0 ≤ uj ≤ vj and vj > 0. There also are M options, each of which includes at least one primary item. A color is assigned to the secondary items of each option; a “blank” color is understood to represent a unique color that appears nowhere else. The task is to find all subsets of options such that

1. each primary item j occurs at least uj times and at most vj times;

2. every secondary item has been assigned at most one color.

Thus every XCC problem is the special case uj = vj = 1 of an MCC problem.

Indeed, the MCC problem is extremely general! For example, its special case uj = 1 and vj = M, without secondary items, is the classical not-necessarily-exact cover problem, in which we simply require each item to appear in at least one option. Section 7.2.2.6 will be entirely devoted to the cover problem.

Let’s confine our attention here to a few more examples of the MCC problem, in preparation for Algorithm M. In the first place, we can tackle a refined version of Wainwright’s partridge puzzle: “Pack at most k squares of size k × k, for 1 ≤ k ≤ n, into an N × N square, without overlapping, so that as many as possible of the N^2 cells are covered.” (As before, N = 1 + 2 + ··· + n.) We know from (6_2) that the entire square can be covered when n = 8; but smaller cases are another story. Solutions for 2 ≤ n ≤ 5 are readily found by hand:

And to prove that every packing for n = 5 must leave at least 13 cells vacant, Algorithm M will show that the MCC problem (61) has no solutions when items #1, #2, #3 are respectively given the multiplicities [0 .. 13], [0 .. 2], [0 .. 3] instead of 1, 2, 3. Exercise 157 constructs optimum packings when n = 6 and n = 7, thereby settling all small cases of the partridge puzzle.

Next, let’s consider an MCC problem of quite a different kind: “Place m queens so that they control all cells of an n×n chessboard.” (The classic 5-queens problem — which should be distinguished from the ‘5 queens problem’ considered earlier — is the special case m = 5, n = 8.) Exercise 7.1.4–241 discusses the history of this problem, which goes back to a remarkable book by de Jaenisch (1863).

We can solve it, MCC-wise, by introducing n^2 + 1 primary items, namely the pairs ij for 0 ≤ i, j < n and the special item #, together with n^2 options:

‘#⁢ ij⁢  i1⁢ j1⁢ i2⁢ j2⁢ .⁢ .⁢ .⁢ itjt’ ⁢    for 0 ≤i, j<n,(64)

where i_1j_1, i_2j_2, ..., itjt are the cells attacked by a queen on ij. Each cell ij is assigned the multiplicity [1 .. m]; item # gets multiplicity m.

From this specification Algorithm M will readily find all 4860 solutions to the 5-queens problem, after 13 gigamems of computation. For example, it begins with 22 ways to cover the corner cell 00. If it puts a queen there, it has 22 ways to cover cell 17; and so on. The branching factor at each step tends to decrease rapidly after three queens have been placed.

The beauty of the MCC setup in (64) is that we can solve many related problems by making simple changes to the specifications. For example, by retaining only the 36 options for 1 ≤ i, j ≤ 6, we could find the 284 solutions that place no queens at the edges of the board. Or by removing the 16 options for 2 ≤ i, j ≤ 5, we would discover that exactly 880 of the solutions place no queens near the middle. Exactly 88 solutions avoid the central two rows and the central two columns. Exactly 200 solutions put all five queens on “black” cells (with i + j even). Exactly 90 avoid the upper left and lower right quadrants. Exactly 2 solutions (can you find them?) place all five queens in the top half of the board.

By changing the multiplicities in the bottom row from [1 .. 5] to 1, we get 18 solutions for which every cell in that row is attacked just once. Or, changing the central 16 multiplicities to [2 .. 5] yields 48 solutions for which every cell near the center is attacked at least twice. Changing all the cell multiplicities to [1 .. 4] reduces the number of solutions from 4860 to 3248; changing them all to [1 .. 3] reduces it to 96. Exercise 161 illustrates several of the less obvious possibilities.

The examples of MCC problems that we’ve seen so far have involved primary items only. Secondary items, and their color controls, add new dimensions and extend the range of applications enormously. Consider, for example, the word rectangles that we investigated briefly in Section 7.2.2. Here’s a 4 × 5 word rectangle that uses only nine distinct letters of the alphabet:

L A B E L
A B I D E .
S L A I N
T E S T S

Can we find one that uses only eight distinct letters, while sticking to common words? (More precisely, is there such a rectangle whose columns are chosen from the most common 1000 four-letter words of English, and whose rows belong to WORDS(2000), the curated collection from the Stanford GraphBase?)

The answer is yes, and in fact there are six solutions:

S T R U TT E A S EE A T E NP R E S SE A S E DA G I L ES E N S EE D G E DW A D E DA R E N AS E E D SH A R S HR A D A RA R E N AS E A T SH A R S HL L A M AE A G E RS T E A MT E S T SS C A R SC O C O AA R R A YR E E D S(66)

One way to find them is to set up an MCC problem in which the primary items are A_0, A_1, A_2, A_3, D_0, D_1, D_2, D_3, D_4, #A, #B, ..., #Z, #; they all have multiplicity 1 except that # has multiplicity 8. There also are secondary items A, B, ..., Z, and ij for 0 ≤ i < 4, 0 ≤ j < 5. The letter-counting is handled by 2 · 26 short options:

‘#A A:0’,‘#A A:1’,‘#B B:0’, ‘#B B:1’; .⁢ .⁢ .⁢ ;‘#Z Z:0’,‘#Z Z:1 #’.(67)

Then each legal 5-letter word c_1c_2c_3c_4c_5 yields four options, ‘Ai i0:c_1 i1:c_2 i2:c_3 i3:c_4 i4:c_5 c_1:1 c_2:1 c_3:1 c_4:1 c_5:1’, for 0 ≤ i < 4; each legal 4-letter word c_1c_2c_3c_4 yields five options, ‘Dj 0j:c_1 1j:c_2 2j:c_3 3j:c_4 c_1:1 c_2:1 c_3:1 c_4:1’, for 0 ≤ j < 5. (Letters that occur more than once in a word are listed only once.)

For example, one of the options chosen for the first solution in (66) is ‘A_3 30:P 31:R 32:E 33:S 34:S P:1 R:1 E:1 S:1’; it forces the options ‘#P P:1 #’, ‘#R R:1 #’, ‘#E E:1 #’, ‘#S S:1 #’ to be chosen too, thus contributing four to the number of chosen options that contain #.

By the way, when Algorithm M is applied to these options, it’s important to use the “nonsharp preference heuristic” discussed in exercise 10 and its answer. Otherwise the algorithm will foolishly make binary branches on the items #A, ..., #Z, before trying out actual words. A 1000-way branch on D_0 is much better than a 2-way branch on #Q, in this situation.

*A new dance step

In order to implement multiplicity, we need to update the data structures in a new way. Suppose, for example, that there are five options available for some primary item p, and suppose they are represented in nodes a, b, c, d, and e. Then p’s vertical list of active options has the following links:

x:pabcdeULINK (x)epabcdDLINK (x)a⁢bcdep(68)

If the multiplicity of p is 3, there are (53)=10 ways to choose three of the five options; but we do not want to make a 10-way branch! Instead, each branch of Algorithm M below chooses only the first of the options that will appear in the solution. Then it reduces the problem recursively; the reduced problem will have a shorter list for p, from which two further options should be selected. Since we must choose either a, b, or c as the first option, the algorithm will therefore begin with a 3-way branch. For example, if b is chosen to be first, the reduced problem will ask for two of options {c, d, e} to be chosen eventually.

The algorithm will recursively find all solutions to that reduced problem. But it won’t necessarily begin by branching again on the same item, p; some other item, q, might well have become more significant. For instance, the choice of b might have assigned color values that make LEN(q) ≤ 1. (The choice of b might also have made c, d, and/or e illegal.)

The main point is that, after we’ve chosen the first of three options for p in the original problem, we have not “covered” p as we did in Algorithms X and C. Item p remains on an equal footing with all other active items of the reduced problem, so we need to modify (68) accordingly.

The operation of reducing the problem by removing an option from an item list, in the presence of multiplicity, is called “tweaking” that option. For example, just after the algorithm has chosen b as the first option for p, it will have tweaked both a and b. This operation is deceptively simple:

tweak (x, p) = {hide'(x) and set  d←DLINK (x), DLINK(p) ←d,ULINK (d)←p, LEN(p)←LEN(p)−1.(69)

(See (51). We will tweak(x, p) only when x = DLINK(p) and p = ULINK(x).) Notice that tweaking x does more than hiding x, but it does less than covering p.

Eventually the algorithm will have tried each of a, b, and c as p’s first option, and it will want to backtrack and undo the tweaking. The actions tweak(a, p), tweak(b, p), tweak(c, p) will have clobbered most of the original uplinks in (68):

x:pabcdeULINK (x)eppppdDLINK (x)d⁢bcdep(70)

Unfortunately, this residual data isn’t sufficient for us to restore the original state, because we’ve lost track of node a. But if we had recorded the value of a when we began, we would be in good shape, because a pointer to node a together with the DLINKs in (70) would now lead us to node b, then to c, and then to d.

Therefore the algorithm maintains an array FT[l], to remember the locations of the “first tweaks” that were made at every level l. And it adds a new dance step, “untweaking,” to its repertoire of link manipulations:

untweak(l) ={Set a⁢←FT[l], p←(a≤N? a:TOP (a)), x←a, y←p;set z←DLINK(p), DLINK(p)←x, k⁢←0;while x⁢≠z,  set ULINK(x)← y⁢  and k←k+1,           unhide'(x), and set y←x, x←DLINK(x);finally set ULINK(z) ←y⁢ and LEN (p)←LEN(p)+ k.(71)

(See exercise 163. This computation relies on a surprising fact proved in exercise 2(a), namely that unhiding can safely be done in the same order as hiding.)

The same mechanism can be used when the specified multiplicity is an interval instead of a single number. For example, suppose item p in the example above is required to occur in either 2, 3, or 4 options, not exactly 3. Then the first option chosen must be a, b, c, or d; and the reduced problem will ask p to occur in either 1, 2, or 3 of the options that remain. Eventually the algorithm will resort to untweaking, after a, b, c, and d have all been tweaked and explored.

Similarly, if p’s multiplicity has been specified to be either 0, 1, or 2, the algorithm below will tweak each of a through e in turn. It will also run though all solutions that omit all of p’s options, before finally untweaking and backtracking.

A special case arises, however, when p’s multiplicity has been specified to be either 0 or 1. In such cases we’re not allowed to choose options b, c, d, or e after option a has been chosen. Therefore it’s important to invoke cover′(p), as in Algorithm C, instead of hiding one option at a time. (See (50).) The individual options of p are then tweaked, to remove them one by one from the active list; this tweaking uses the special operation tweak′(x, p), which is like tweak(x, p) in (69) except that it omits the operation hide′(x), because hiding was already done when p was covered. Finally, the case of 0-or-1 multiplicity eventually concludes by invoking the routine untweak′(l), which is like untweak(l) in (71) except that (i) it omits unhide′(x), and (ii) it calls uncover′(p) after restoring LEN(p).

We’re ready now to write Algorithm M, except that we need a way to represent the multiplicities in the data structures. For this purpose every primary item has two new fields, SLACK and BOUND. Suppose the desired multiplicity of p is in the interval [u..v;], where 0 ≤ u ≤ v and v ≠ 0; Algorithms X and C correspond to the case u = v = 1. Then we set

SLACK(p)   ←v⁢  − u,BOUND(p) ←v(72)

when the algorithm begins. The value of SLACK(p) will never be changed. But BOUND(p) will decrease dynamically, as we reduce the problem, so that we will never choose more options for p than its current bound.

Algorithm M (Covering with multiplicities and colors). This algorithm visits all solutions to a given MCC problem, extending Algorithms X and C.

M1. [Initialize.] Set the problem up in memory as in step C1 of Algorithm C, with the addition of multiplicity specifications (72). Also set N to the number of items, N_1 to the number of primary items, Z to the last spacer address, and l ← 0.

M2. [Enter level l.] If RLINK(0) = 0 (hence all items have been covered), visit the solution that is specified by x_0x_1 ... xl−1 and go to M9. (See exercise 164.)

M3. [Choose i.] At this point the items i_1, ..., it still need to be covered, where i_1 = RLINK(0), ij+1 = RLINK(ij), RLINK(it) = 0. Choose one of them, and call it i. (The MRV heuristic of exercise 166 often works well in practice.) If the branching degree θi is zero (see exercise 166), go to M9.

M4. [Prepare to branch on i.] Set xl ← DLINK(i) and BOUND(i) ← BOUND(i)−1. If BOUND(i) is now zero, cover item i using (50). If BOUND(i) ≠ 0 or SLACK(i) ≠ 0, set FT[l] ← xl.

M5. [Possibly tweak xl.] If BOUND(i) = SLACK(i) = 0, go to M6 if xl ≠ i, to M8 if xl = i. (That case is like Algorithm C.) Otherwise if LEN(i) ≤ BOUND(i) − SLACK(i), go to M8 (list i is too short). Otherwise if xl ≠ i, call tweak(xl,i) (see (69)), or tweak′(xl,i) if BOUND(i) = 0. Otherwise if BOUND(i) ≠ 0, set p ← LLINK(i), q ← RLINK(i), RLINK(p) ← q, LLINK(q) ← p.

M6. [Try xl.] If xl ≠ i, set p ← xl + 1, and do the following while p ≠ xl: Set j ← TOP(p); if j ≤ 0, set p ← ULINK(p); otherwise if j ≤ N_1, set BOUND(j) ← BOUND(j) − 1, p ← p + 1, and cover′(j) if BOUND(j) is now 0; otherwise commit(p, j) and set p ← p + 1. (This loop covers or commits the items ≠ i in the option that contains xl.) Set l ← l + 1 and return to M2.

M7. [Try again.] Set p ← xl − 1, and do the following steps while p ≠ xl: Set j ← TOP(p); if j ≤ 0, set p ← DLINK(p); otherwise if j ≤ N_1, set BOUND(j) ← BOUND(j) + 1, p ← p − 1, and uncover′(j) if BOUND(j) is now 1; otherwise uncommit(p, j) and set p ← p − 1. (This loop uncovers or uncommits the items ≠ i in the option that contains xl, using the reverse order.) Set xl ← DLINK(xl) and return to M5.

M8. [Restore i.] If BOUND(i) = SLACK(i) = 0, uncover item i using (52). Otherwise call untweak(l) (see (71)), or untweak′(l) if BOUND(i) = 0. Set BOUND(i) ← BOUND(i) +1.

M9. [Leave level l.] Terminate if l = 0. Otherwise set l ← l − 1. If xl ≤ N, set i ← xl, p ← LLINK(i), q ← RLINK(i), RLINK(p) ← LLINK(q) ← i, and go to M8. (That reactivates i.) Otherwise set i ← TOP(xl) and go to M7.

*Analysis of Algorithm X

Now let’s get quantitative, and see what we can actually prove about the running time of these algorithms.

For simplicity, we’ll ignore color constraints and look only at Algorithm X, as it finds all solutions to an exact cover problem, where the problem is specified in terms of an M × N matrix A of 0s and 1s such as (5).

We’ll assume that the problem is strict, in the sense that no two rows of the matrix are identical, and no two columns of the matrix are identical. For if two or more rows or columns are equal, we need keep only one of them; it’s easy to relate all solutions of the original problem A to the solutions of this reduced problem A′. (See exercise 179.)

Our first goal will be to find an upper bound on the number of nodes in the search tree, as a function of the number of rows of A (the number of options). This upper bound grows exponentially, because the exact cover problem can have lots of solutions; but we’ll see that it can’t actually be extremely large.

For this purpose we’ll define the doomsday function D(n), which will have the property that the search tree for every strict exact cover problem with n options has at most D(n) nodes, when Algorithm X uses the MRV heuristic in step X3.

The search tree has a root node labeled with the original matrix A, and its other nodes are defined recursively: When a node at level l is labeled with a subproblem for which step X3 makes a t-way branch, that node has t subtrees, whose roots are labeled by the reduced problem that remains after step X4 has covered item i and after step X5 has optionally covered one or more other items j, for t different choices of xl.

Here, for example, is the complete search tree when A is the matrix of (5):

(Each matrix and submatrix in this diagram has been framed with a light-gray border. The node at bottom left illustrates a 0×1 submatrix, where the algorithm had to backtrack because it had no way to cover the remaining column. The node at bottom right illustrates a 0×0 submatrix, which happens to be a solution to the 1 × 2 problem above it.) We can, if we like, reduce all of the submatrices by eliminating repeated columns, although Algorithm X doesn’t do this; then we get strict exact cover problems at every node of the search tree:

A t-way branch implies that the matrix A has a certain structure. We know that there’s some column, say i_1 = i, that has 1 in exactly t rows, say o_1, ..., ot, and that every column contains at least t 1s. When we branch on row op, for 1 ≤ p ≤ t, the reduced problem that defines the pth subtree will retain all but sp of the rows of A, where sp is the number of rows that intersect op. We can order the rows so that s_1 ≤ ··· ≤ st. For example, in (73) we have t = 2 and s_1 = s_2 = 4.

A nice thing now happens: There’s always a unique index 0 ≤ t′ ≤ t such that
sp=t+p−1,for 1 ≤p⁢≤t′.(74)

That is, either s_1 > t and t′ = 0; or s_1 = t and t′ = 1 and either (t = 1 or s_2 > t + 1); or s_1 = t and s_2 = t + 1 and t′ = 2 and either (t = 2 or s_3 > t + 2); or ... ; or s_1 = t and s_2 = t + 1 and ... and st = 2t − 1 and t′ = t.

Suppose, for example, that t = 4 and s_1 = 4; we must prove that s_2 ≥ 5. Since s_1 = 4, row o_1 doesn’t intersect any rows except {o_1, ..., ot}; consequently option o_1 consists of the single item ‘i_1’. Hence option o_2 must contain at least two items, ‘i_1 i_2 ...’, otherwise the problem wouldn’t be strict. This new item appears in at least 4 options, however, one of which is different from o_1. Option o_2 therefore intersects 5 or more options (including itself). QED.

Similarly, if t = 4 and s_1 = 4 and s_2 = 5, exercise 180 proves that s_3 ≥ 6, and indeed it proves that even more is true. For example, if t = t′ = 4, so that (s_1, s_2, s_3, s_4) = (4, 5, 6, 7) as demanded by (74), exercise 180 proves the existence of options o_5, o_6, o_7 that have a particularly simple form:

o1⁢ = ‘i1’;o2⁢=‘i1⁢ i2’; o3=‘i1⁢ i2⁢ ⁢i3’;o4=‘i1⁢ i2⁢ ⁢i3 ⁢i4’; ⁢o5=‘i2⁢ i3⁢ ⁢i4 ⁢.⁢ .⁢ .⁢ ’;  o6=‘i3⁢ ⁢ ⁢i4 ⁢.⁢ .⁢ .⁢ ’;o7=‘⁢ ⁢i4 ⁢.⁢ .⁢ .⁢ ’.(75)

Okay, we’re ready now to construct the promised “doomsday function” D(n). It starts out very tame,

D(0)=D(1)=1;(76)

and for convenience we set D(n) = −∞ if n < 0. When n ≥ 2 the definition is

D(n)=max{d(n,t,t′) | 1≤t<n⁢ and 0≤t′≤t},(77)

where d(n, t, t′) is an upper bound for the size of the search tree over all n-option strict exact cover problems whose parameters (74) are t and t′. One such bound,
d(n,t,0)=1+t⋅D(n−t−1),(78)

handles the case t′ = 0, because the search tree in that case is a t-way branch

and each subproblem Ap has at most n − t − 1 options.

The formula for t′ > 0 is more intricate, and less obvious:

d(n,t,t′)=t′+t′⋅D(n−t−t′+1)+(t−t′)⋅D(n−t−t′−1),  if 1≤t′≤t.(80)

It can be justified by the structure theory of exercise 180, using the fact that each of the first t′ − 1 branches is immediately followed by a 1-way branch. For example, the search tree looks like this when t = 5 and t′ = 3:

Here A1′ is the only way to cover i_2 in A_1, and A2′ is the only way to cover i_3 in A_2. The strict problems A1′, A2′, and A_3 have at most n − 7 options; A_4 and A_5 have at most n − 9. Therefore (81) has at most 3 + 3D(n − 7)+2D(n − 9) nodes.

With an easy computer program, (76), (78), and (80) lead to the values

n

=

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

D(n)

=

1

1

2

4

5

6

10

13

17

22

31

41

53

69

94

125

165

213

283

377

501

661

and it turns out that the maximum is attained uniquely when t = 4 and t′ = 0, for all n ≥ 19. Hence we have D(n) = 1+4D(n − 5) for all sufficiently large n; and in fact exercise 181 exhibits a simple formula that expresses D(n) exactly.

Theorem E. The search tree of a strict exact cover problem with n options has O(4n/5) = O(1.31951n) nodes; it might have as many as Ω(7n/8) = Ω(1.27537n).

Proof. The upper bound follows from exercise 181; the lower bound follows from the family of problems in exercise 182.

[David Eppstein presented this theorem to the author as a birthday greeting(!); see 11011110.github.io/blog/2008/01/10/analyzing-algorithm-x.xhtml.]

So far we’ve simply been analyzing the number of nodes in Algorithm X’s search tree. But some nodes might cost much more than others, because they might remove unusually many options from the currently active lists.

Therefore let’s probe deeper, by studying the number of updates that Algorithm X makes to its data structures, namely the number of times that it uses operation (1) to remove an element from a doubly linked list. (This is also the number of times that it will eventually use operation (2) to restore an element.) More precisely, the number of updates is the number of times cover(i) is called, plus the number of times that hide(p) sets LEN(x) ← LEN(x) − 1. (See (12) and (13).) The total running time of Algorithm X, measured in mems, usually turns out to be roughly 13 times the total number of updates that it makes.

It’s instructive to analyze the number of updates that are made when solving the “extreme” exact cover problems, which arise when there are n items and 2n − 1 options: Such problems have the most solutions and the most data, because every nonempty subset of the items is an option. The solutions to these extreme problems are precisely the set partitions — the ϖn possible ways to partition the items into disjoint blocks, which we studied in Section 7.2.1.5. For example, when n = 3 the options are ‘1’, ‘2’, ‘1 2’, ‘3’, ‘1 3’, ‘2 3’, ‘1 2 3’, and there are ϖ_3 = 5 solutions: ‘1’, ‘2’, ‘3’; ‘1’, ‘2 3’; ‘1 2’, ‘3’; ‘1 3’, ‘2’; ‘1 2 3’.

Any given item can be covered in 2n−1 ways; and if we cover it with an option of size k, we’re left with the extreme problem on the remaining n − k items. Algorithm X therefore advances 2n−1 times from level 0 to level 1, after which it essentially calls itself recursively. And at level 0 it performs a certain number of updates, say υn, regardless of what strategy is used in step X3 to choose an item for branching. Therefore it makes a total of xn updates, where
xn=vn+(n−10)xn−1+(n−11)xn−2+⋅⋅⋅+(n−1n−1)x0.(82)

The solution to this recurrence is x_0 = v_0, x_1 = v_0 + v_1, x_2 = 2v_0 + v_1 + v_2, and in general xn=∑k=0nank⁢ vk, where the matrix (ank) is

(1000000.⁢ .⁢ .1100000.⁢.⁢.2110000.⁢.⁢.5311000.⁢.⁢.15941100.⁢.⁢.5231145110.⁢.⁢.2031215420611.⁢.⁢..⁢......)=(  1000000.⁢.⁢.⁢−1100000.⁢.⁢.−1−110000.⁢.⁢ .−1−2−11000.⁢.⁢.−1−3−3−1100.⁢.⁢.−1−4−6−4−110.⁢.⁢.−1−5−10−10−5−11.⁢.⁢..⁢......)−1,(83)

with rows and columns numbered from 0. The numbers an0 in the left column, which solve (82) when vn = δn0, are the familiar Bell numbers ϖn; they enumerate the leaves in the search tree. The numbers an1 in the next column, which solve (82) when υn = δn1, are the Gould numbers ϖ^n; they enumerate set partitions whose last block or “tail” is a singleton, when the blocks of the partition are ordered by their least elements. In general, ank for k > 0 is the number of set partitions whose tail has size k. [See H. W. Gould and J. Quaintance, Applicable Analysis and Discrete Mathematics 1 (2007), 371–385.]

Exercise 186 proves that the actual number of updates at level 0 is
υn=((9n−27)4n−(8n−32)3n+(36n−36)2n+72−41δn⁢0)/72;(84)

and exercise 187 exploits relationships between the sequences ⟨ank⟩ to show that
xn=22ϖn+12⁢ ϖ^n−(23n−1)3n−52n2n−12n−5−12δn1−18δn0.(85)

Asymptotically, ϖ^n/ϖn converges rapidly to the “Euler–Gompertz constant”

g^=∫0∞  e−xdx1+x=0.59634⁢  73623⁢ 23194⁢ 07434⁢ 10784⁢ 99369⁢ 27937⁢ 60741+(86)

(see exercise 189). Thus xn ≈ (22 + 12ĝ)ϖn ≈ 29.156ϖn, and we’ve proved that Algorithm X performs approximately 29.156 updates per solution to the extreme exact cover problem, on average. That’s encouraging: One might suspect that the list manipulations needed to deal with 2n options of average length n would cost substantially more, but the dancing-links approach turns out to be within a constant factor of Section 7.2.1.5’s highly tuned methods for set partitions.

*Analysis of matching problems

Among the simplest exact cover problems are the ones whose options don’t contain many items. For example, a so-called X2C problem (“exact cover with 2-sets”) is the special case where every option has exactly 2 items; an X3C problem has 3 items per option; and so on. We’ve seen in (30) that sudoku is an X4C problem.

Let’s take a close look at the simplest case, the X2C problems. Despite their simplicity, we’ll see that such problems actually include many cases of interest. Every X2C problem corresponds in an obvious way to a graph G, whose vertices υ are the items and whose edges u — υ are the pairs of items that occur together in an option ‘u υ’. In these terms the X2C problem is the classical task of finding a perfect matching, namely a set of edges that contains each vertex exactly once.

We shall study efficient algorithms for perfect matching in Section 7.5.5 below. But an interesting question faces us now, in the present section: How well does our general-purpose Algorithm X compare to the highly tuned special-purpose algorithms that have been developed especially for matching in graphs?

Suppose, for example, that G is the complete graph K2q+1. In other words, suppose that there are n = 2q + 1 items {0, 1, . . . , 2q}, and that there are m=(2q+12)=(2q+1)q options ‘i j’ for 0 ≤ i < j ≤ 2q. This problem clearly has no solution, because we can’t cover an odd number of points with 2-element sets! But Algorithm X won’t know this (unless we give it a hint by factoring the problem appropriately). Thus it’s interesting to see how long Algorithm X will spin its wheels before giving up on this problem.

In fact the analysis is easy: No matter what item i is chosen in step X3, the algorithm will split nicely into 2q branches, one for each option ‘i j’ with j ≠ i. And each of those branches will be equivalent to the matching problem on the remaining 2q − 1 items; the remaining options will, in fact, be equivalent to the complete graph K2q−1. Thus the search tree will have 2q nodes at depth 1, (2q)(2q − 2) nodes at depth 2, ..., and (2q)(2q − 2) ... (2) = 2qq! nodes at depth q. Backtracking will occur at the latter nodes, which are leaves because they correspond to impossible matching in the graph K_1.

How long does this process take? A closer look (see exercise 193) shows that the total number of updates to the data structure will satisfy the recurrence

U(2q+1)=1+2q+4q2+2qU(2q−1),for q > 0;       U(1)=1.(87)

Consequently (see exercise 194) the number of updates needed by Algorithm X to discover that K2q+1 has no perfect matching turns out to be less than 8.244 times the number of leaves.

There’s better news when Algorithm X is presented with the complete graph K_2q, because this problem has solutions — lots of them. Indeed, it’s easy to see that K_2q has exactly (2q − 1)(2q − 3) ... (3)(1) = (2q)!/(2qq!) perfect matchings. For example, K_8 has 7 · 5 · 3 · 1 = 105 of them. The total number of updates in this case satisfies a recurrence similar to (87):

U(2q)=1−2q+4q2+(2q−1)U(2q−2),         for q>0;    U(0)=0.(88)

And exercise 194 proves that this is less than 10.054 updates per matching found.

Armed with these facts, we can work out what happens when the graph

is presented to Algorithm X. (This graph has 2q + 2r + 2 vertices.) The result, which is revealed in exercise 195, is both instructive and bizarre.

A 2D matching problem — also called bipartite matching, and 2DM for short — is the special case of an X2C problem in which every option has the form ‘Xj Yk’, where the items {X_1, ..., Xn} and {Y_1, ..., Yn} are disjoint sets. Higher-dimensional matching is defined similarly; sudoku is actually a case of 4DM.

Let’s round out our analyses of matching by considering the bounded permutation problem: “Given a sequence of positive integers a_1 ... an, find all permutations p_1 ... pn of {1, ..., n} such that pj ≤ aj for 1 ≤ j ≤ n.” We can assume that a_1 ≤ ··· ≤ an, because p_1 ... pn is a permutation; we can also assume that aj ≥ j, otherwise there are no solutions; and we can assume without loss of generality that an ≤ n. This is easily seen to be a 2DM problem, having exactly a_1 + ··· + an options, namely ‘Xj Yk’ for 1 ≤ j ≤ n and 1 ≤ k ≤ aj.

Suppose we branch first on X_1. Then each of the a_1 subproblems is easily seen to be essentially a bounded permutation problem with n decreased by 1, and with a_1 ... an replaced by (a_2 − 1) ... (an − 1). Thus a recursive analysis applies, and again we find that the dancing links algorithm does rather well. For example, if aj = min(j + 1, n) for 1 ≤ j ≤ n, there are 2n−1 solutions, and Algorithm X performs only about 12 updates per solution. If aj = min(2j, n) for 1 ≤ j ≤ n, there are ⌊n+12⌋! ⌊n+22⌋! solutions, and Algorithm X needs only about 4e − 1 ≈ 9.87 updates per solution. Exercise 196 has the details.

*Maintaining a decent focus

Some backtrack algorithms waste time by trying to solve two or more loosely related problems at once. Consider, for example, the 2DM problem with 7 items {0, 1, ..., 6} and the following 13 options:

‘0⁢ ⁢ 1’, ‘0⁢ ⁢  2’,‘⁢1⁢ ⁢ 4’,‘⁢1⁢ ⁢ 5’,‘1⁢ ⁢ 6’,‘2⁢  4’,‘2⁢ ⁢ 5’,‘2⁢  6’,‘3⁢   4’,‘3⁢  5’,‘3⁢   6’,‘4⁢  5’,‘4⁢  6’.(90)

Algorithm X, using its MRV heuristic, will branch on item 0, choosing either ‘0 1’ or ‘0 2’; then it will be faced with a three-way branch; and it will eventually conclude that there’s no solution, after implicitly traversing a 19-node search tree,

We get an extreme example of bad focus if we take n independent copies of problem (90), with 7n items {k0, k1, ..., k6} and 13n options ‘k0 k1’, ‘k0 k2’, ..., ‘k4 k6’, for 0 ≤ k < n: The algorithm will begin with 2-way branches on each of 00, 10, ..., (n−1)0; then it will show that each of the 2n resulting subproblems is unsolvable, making 3-way branches as it begins to study each one. Its search tree, before giving up, will have 10 · 2n − 1 nodes. By contrast, if we had somehow forced the algorithm to keep its attention on the very first copy of (90) (the case k = 0), instead of using the MRV heuristic, it would have concluded that there are no solutions after backtracking through only 19 nodes.

Similarly, the simple exact cover problem on items {0, 1, ..., 5} with options

‘0⁢ ⁢ 1’, ‘0⁢ ⁢  2’,‘⁢1⁢ 3⁢ ⁢ 4’,‘⁢1⁢ 3⁢ ⁢ 5’,‘⁢1⁢ 4⁢ ⁢ 5’,‘⁢2⁢ 3⁢  4’,‘2⁢  3⁢ ⁢4⁢ 5’,‘2⁢  4⁢ 5’,‘3⁢ ⁢ 4’,‘3⁢  5’,‘4⁢  5’,(92)

has a search tree with 9 nodes, one of which is a solution:

Taking n independent copies of (92) gives us an exact cover problem with a unique solution, whose search tree via Algorithm X and MRV has 8 · 2n − 7 nodes. But if the algorithm had been able to focus on one problem at a time, it would have discovered the solution with a search tree of only 8n + 1 nodes.

From a practical standpoint, it must be admitted that the exponential behavior of these badly focused toy examples is worrisome only when n is larger than 30 or so, because 2n is not scary for modern computers when n is small. Still, we can see that a well-focused approach can give significant advantages. So it will be useful to understand how Algorithm X and its cousins behave in general, when the input actually consists of two independent problems.

Let’s pause a minute to define the search tree precisely. Given an m × n matrix A of 0s and 1s, the search tree T of its associated exact cover problem is simply a solution node ‘▪’ when n = 0; otherwise T is

where the item chosen for branching in step X3 has d options, and Tk is the search tree for the reduced problem after the items of the kth option have been removed. (With the MRV heuristic, d is the minimum length of all active item lists, and we choose the leftmost item having this value of d.)

The exact cover problem that we get when trying to solve two independent problems given by matrices A and A′ is the problem that corresponds to the direct sum A ⨁ A′ (see Eq. 7–(40)). Therefore if T and T′ are the corresponding search trees, we will write T ⨁ T′ for the search tree of A ⨁ A′, under the MRV heuristic. (That tree depends only on T and T′, not on any other aspects of A or A′.) If either T or T′ is simply a solution node, the rule is simple:

T⊕ ▪=T;▪⊕  T′=T′.(95)

Otherwise T and T′ have the form (94), and we have

Dear reader, please work exercise 202 — which is very easy! — before reading further. That exercise will help you to understand the definition of T ⨁ T′; and you’ll also see that every node of T ⨁ T′ is associated with an ordered pair αα′, where α and α′ are nodes of T and T′, respectively. These ordered pairs are the key to the structure of T ⨁ T′: If α and α′ appear at levels l > 0 and l′ > 0 of their trees, so that they are reached from the roots by the respective paths α_0 ─ α_1 ─ ··· ─ αl = α and a1′—a1′—⋅⋅⋅al′′=a′, then the parent of αα′ in T ⨁T′ is either αl−1α′ or aal′−1′. Consequently every ancestor αk of α in T , for 0 ≤ k ≤ l, occurs in an ancestor ak⁢ ak′′ of αα′ in T ⨁ T′, for some 0 ≤ k′ ≤ l′.

Let deg(α) be the number of children that node α has in a search tree, except that we define deg(α) = ∞ when α is a solution node. (Equivalently, deg(α) is the minimum length of an item list, taken over all active items in the subproblem that corresponds to node α. If α is a solution, there are no active items, hence the minimum is infinite.) Let’s call α a dominant node if its degree exceeds the degree of all its proper ancestors. The root node is always dominant, and so is every solution node. For example, (91) has three dominant nodes, and (93) has four.

In these terms, exercise 205 proves a significant fact about direct sums:

Lemma D. Every node of T ⨁ T′ corresponds to an ordered pair αα′ of nodes belonging to T and T′. Either α or α′ is dominant in its tree, or both are.

Lemma D is good news, focuswise, because the search trees that arise in practice tend to have comparatively few dominant nodes. In such cases the MRV heuristic manages to keep the search reasonably well focused, because T ⨁ T′ isn’t too large. For example, the search trees for Langford pairs, or for the n queens problem, are “minimally dominant”: Only their root nodes and their solutions dominate; elsewhere the branching degrees don’t reach new heights.

Fig. 73. A 15 × 15 square can be tiled with Y pentominoes, by setting up an exact cover problem with one item for each cell and one option for each placement of a Y. (To eliminate the 8-fold symmetry, only 5 of the 40 options for occupying the center cell were permitted.) Algorithm X’s first solution, shown here, was found by branching successively on the possible ways to fill the cells marked 0, 1, ..., 9, a, ..., z, A, ..., I.

Let’s look now at a non-contrived example. Figure 73 illustrates a somewhat surprising way to pack 45 Y pentominoes into a 15 × 15 square. [Such tilings were first found in 1973 by J. Haselgrove, at a time when perfect Y-packings were known only for rectangles whose area was even. Her program first ruled out all rectangles of odd area less than 225, as well as the case 9×25, before discovering a 15×15 solution. See JRM 7 (1974), 229.] Notice that the first eight pentominoes in Fig. 73, those marked 0 through 7, were placed in or next to the four corners — thus flirting dangerously with the possibility that the algorithm might be trying to solve four independent problems at once! Luckily, the subsequent choices were able to gain and retain focus, because hard-to-fill cells almost always kept popping up near the recent activity. A five-way branch was needed only when placing the pentominoes marked 8, b, e, g, h, and C in the solution shown.

Focus can sometimes be improved by explicitly preferring some items to others, based on their names; see the “sharp preference” heuristic of exercise 10.

Exercise 207 discusses another approach, an experimental modification of Algorithm X, which attempts to improve focus in situations like Fig. 73 by allowing a user to specify the importance of recent activity. The ideas are interesting, but so far they haven’t led to any spectacular successes.

Exploiting local equivalence

A close look at Fig. 73 reveals another phenomenon that is often present in exact cover problems: The tiles marked 8 and b, near the upper right corner, form an ‘H’ shape, which could be reflected left-right to yield another valid tiling. In fact there are three other such H’s in the picture; therefore Fig. 73 actually represents 2^4 = 16 different solutions to the problem, although those solutions are “locally” equivalent.

It turns out that the 15 × 15 tiling problem in Fig. 73 has exactly 212 mutually incongruent solutions, each of which can be rotated and/or reflected to make a set of eight that are congruent to each other; and each of those solutions contains at least two H’s. Algorithm X needs just 92 Gμ to find them all. But we can do even better, because of H-equivalence: A slight extension to the options of the exact cover problem will produce only the solutions for which every ‘H’ has just one of its two forms — and so does every , namely every 90° rotation of an ‘H’. (See exercise 208.) This modified problem has just 16 solutions, which are obtained with only 26 Gμ of computation and compactly represent all 212.

In general, an exact cover might contain four distinct options α, β, α′, β′ for which α and β are disjoint, α′ and β′ are disjoint, and
α+β=α′+β′.(97)

(The ‘+’ sign here is like ‘∪’; it stands for addition of binary vectors, when options are rows of a 0–1 matrix.) In such cases we say that (α, β; α′, β′) is a bipair. Whenever (α, β; α′, β′) is a bipair, every solution that contains both α and β leads to another solution that contains both α′ and β′, and vice versa. Thus we can avoid considering half of all such solutions if we exclude one of those alternatives. And it’s easy to do that: For example, to exclude all cases that contain both α′ and β′, we simply introduce a new secondary item, and append it to options α′ and β′.

To illustrate this idea, let’s apply it to the unsolvable toy problem (90). That problem has many bipairs, but we’ll consider only two of them,

(‘0 1’,‘2 4’;‘0 2’,‘1 4’)and(‘0 1’,‘2 5’;‘0 2’,‘1 5’).(98)

To avoid solutions that contain both ‘0 1’ and ‘2 4’, as well as those that contain both ‘0 2’ and ‘1 5’, we introduce secondary items A and B, and we extend four of the options (90) to

‘0 1 A’,‘0 2 B’,‘1 5 B’,‘2 4 A’.(99)

Then the search tree (91) reduces to

and the former focusing problems disappear.

But wait, you say. Both of the bipairs in (98) involve the options ‘0 1’ and ‘0 2’. Why is it legal to prefer different halves of those overlapping bipairs? Isn’t it possible that we might “paint ourselves into a corner,” if we allow ourselves to make arbitrary decisions about each of several interrelated bipairs?

That’s a good question. Indeed, bad decisions can lead to trouble. Consider, for example, the problem of perfect matching on the complete bipartite graph K3,3, which can be coded as an X2C with the nine options ‘x X’ for x ∈ {x, y, z} and X ∈ {X, Y, Z}. (The problem of perfect matching on Kn,n is equivalent to finding the permutations of n elements; thus K3,3 has 3! = 6 perfect matchings.)

Every bipair (‘t u’, ‘v w’; ‘t w’, ‘u v’) in a perfect matching problem is equivalent to a 4-cycle t ─ u ─ v ─ w ─ t in the given graph. And if we disallow the right halves of the six bipairs

(‘x Y’,‘y X’;‘x X’,‘y Y’)(‘x Y’,‘y Z’;‘x Z’,‘y Y’)(‘y Y’,‘z X’;‘y X’,‘z Y’)(‘y Y’,‘z Z’;‘y Z’,‘z Y’)(‘z Y’,‘x X’;‘z X’,‘x Y’)(‘z Y’,‘x Z’;‘z Z’,‘x Y’)

we obtain nine options that have no solution:

‘x X A’,‘y X B’,‘z X C’,‘x Y C D’‘y Y A E’‘z Y B F’,‘x Z E’,‘y Z F’,‘z Z D’.(101)

Fortunately, however, there’s always a safe and easy way to proceed. We can assign an arbitrary (but fixed) ordering to the set of all options. Then, if for every bipair (α, β; α′, β′) we always choose the half that contains min(α, β, α′, β′), the choices will be consistent.

More precisely, we can express every bipair in the canonical form

(α,β;α′,β′)α<β,α<α′, and α′<β′,(102)

with respect to any fixed ordering of the options. An exact covering is called strong, with respect to a set of such canonical bipairs, if its options don’t include both α′ and β′ for any bipair in that set.

Theorem S. If an exact cover problem has a solution, it has a strong solution.

Proof. Every solution ∑ corresponds to a binary vector x = x_1 ... xM , where xj = [option j is in ∑]. If ∑ isn’t strong, with respect to a given set of canonical bipairs, it violates at least one of those bipairs, say (α, β; α′, β′). Thus there are indices j, k, j′, k′ with j < k, j < j′, and j′ < k′ such that α, β, α′, β′ are respectively the jth, kth, j′th, k′th options, and such that xj′ = xk′ = 1. By (97), xj = xk = 0; and we obtain another solution ∑′ by setting xj′←xk′←1, xj′′←xk′′←0, otherwise xi′=xi. This vector x′ is lexicographically greater than x; so we’ll eventually obtain a strong solution by repeating the process.

In particular, we’re allowed to exclude both ‘0 1’ and ‘2 4’, as well as both ‘0 2’ and ‘1 5’, with respect to the bipairs (98), because we can choose an ordering in which options ‘1 4’ and ‘2 5’ precede the other options ‘0 1’, ‘0 2’, ‘1 5’, ‘2 4’.

Another convenient way to make consistent choices among related bipairs is based on ordering the primary items, instead of the options. (See exercise 212).

It’s interesting to apply this theory to the problem of perfect matching in the complete graph K2q+1. We showed in (87) above that Algorithm X needs a long time — Ω(2qq!) mems — to discover that this problem has no solution. But bipairs come to the rescue.

Indeed, K2q+1 has lots of bipairs, Θ(q^4) of them. A straightforward application of Theorem S, using the natural order ‘0 1’ < ‘0 2’ < ··· < ‘(2q−1) 2q’ on the (2q+12) options, solves the problem in Θ(q^4) mems, by using just Θ(q^3) of the bipairs. And a more clever way to order the options allows us to solve it in only Θ(q^2) mems, using just Θ(q^2) well-chosen bipairs. The search tree can in fact be reduced to just 2q + 1 nodes — which is optimum! Exercise 215 explains all.

*Preprocessing the options

Sometimes the input to an XCC problem can be greatly simplified, because we can eliminate many of its options and/or items. The general idea of “preprocessing,” which transforms one combinatorial problem into an equivalent but hopefully simpler one, is an important paradigm, which is often called kernelization for reasons that we shall discuss later.

Algorithm P below is a case in point. It takes any sequence of items and options that would be acceptable to Algorithm X or to Algorithm C, and produces another such sequence with the same number of solutions. Any solution of the new problem can in fact be converted to a solution of the original one, if desired.

The algorithm is based entirely on two general principles, used repeatedly until they no longer apply:

- An option can be removed if it blocks all uses of some primary item.

- An item can be removed if some primary item always forces it to appear.

More precisely, let o be a generic option ‘i_1 i_2[:c_2] ... it[:ct]’, where i_1 is primary and the other t − 1 items might have color controls. When Algorithm C deals with option o, it covers i_1 in step C4 and commits the other items in step C5, thereby removing all options that aren’t compatible with o. If this process causes some primary item p to lose its last remaining option, we say that p is “blocked” by o. In such a case o is useless, and we can remove it. For example, ‘deg’ can be removed from (6), because it blocks a;‘1 s_4 s_6’ can be removed from (17), because it blocks s_3; then ‘1 s_1 s_3’ and ‘2 s_2 s_5’ also go away, because they block s_4.

That was the first principle mentioned above, the one that removes options. The item-removing principle is similar, but more dramatic when it applies: Let p be a primary item, and suppose that p’s options all contain an uncolored instance of some other item, i. In such a case we say that p “forces” i; and we can remove item i, because p must be covered in every solution and it carries i along. For example, a forces d in (6). Hence we can remove item d, shortening the second and fourth options to just ‘a g’ and ‘a f’.

These two principles, blocking and forcing, are by no means a complete catalog of transformations that could be used to preprocess exact cover problems. For example, they are incapable of discovering the fact that (38) is a useless option in the pentomino problem, nor do they discover the simplifications that we deduced by factoring the Soma cube problem. (See the discussion before (46).) Exercise 219 discusses yet another way to discard superfluous options.

A “perfect” and “complete” preprocessor would in fact be able to recognize any problem that has at most one solution. We can’t hope to achieve that, so we’ve got to stop somewhere. We shall limit ourselves to the removal of blocking and forcing, because those transformations can be done in polynomial time, and because no other easily recognizable simplifications are apparent.

Algorithm P discovers all such simplifications by systematically traversing the given items and options, using the same data structures that were enjoyed by Algorithm C. It cycles through all items i, trying first to remove i by studying what happens when i is covered. If that fails, it studies what happens when the items of options that begin with i are committed. It needs some small variations of the former ‘cover’ and ‘hide’ operations (compare with (12)–(15), (50)–(53)):

cover″(i)={Set p←DLINK(i).  While p≠i,    hide″(i) unless COLOR(p)≠0,    then set p←DLINK(p) and repeat.(103)

hide″(p)={Do operation hide(p); but also set S←x,   whenever LEN(x) has been set to 0 and x≤N1.(104)

uncover″(i)={Set p←ULINK(i).  While p≠i,    unhide(i) unless COLOR(p)≠0,    then set p←ULINK(p) and repeat.(105)

Algorithm P (Preprocessing for exact covering). This algorithm reduces a given XCC problem until no instances of blocking or forcing are present. It uses the data structures of Algorithm C, together with new global variables C and S.

P1. [Initialize.] Set the problem up in memory, as in step C1 of Algorithm C. (Again there are N items, of which N_1 are primary.) Also set C ← 1. If there’s an item i ≤ N_1 with LEN(i) = 0, go to P9.

P2. [Begin a round.] If C = 0, go to P10. Otherwise set C ← 0, i ← 1.

P3. [Is item i active?] If i = N, return to P2. Otherwise if LEN(i) = 0, go to P8.

P4. [Cover i.] Set S ← 0. Use (103) to cover item i. Then go to P7 if S ≠ 0; otherwise set x ← DLINK(i).

P5. [Try x.] If x isn’t the leftmost remaining node of its option, go to P6. Otherwise use the method of exercise 220 to test whether this option blocks some primary item. If so, set C ← 1, TOP(x) ← S, and S ← x.

P6. [Try again.] Set x ← DLINK(x), and return to P5 if x ≠ i. Otherwise uncover item i using (105); use the method of exercise 221 to delete all options that were stacked in step P5; and go to P8.

P7. [Remove item i.] Uncover item i (which is forced by the primary item S). Then use the method of exercise 222 to delete or shorten every option that uses item i. Finally, set C ← 1, DLINK(i) ← ULINK(i) ← i, LEN(i) ← 0.

P8. [Loop on i.] Set i ← i + 1 and return to P3.

P9. [Collapse.] Set N ← 1 and delete all options. (The problem is unsolvable.)

P10. [Finish.] Output the reduced problem, whose items are those for which LEN(i) > 0 or i = N = 1, and terminate. (See exercise 223.)

How effective is Algorithm P? Well, sometimes it spins its wheels and finds absolutely nothing to simplify. For example, the options (16) for n Langford pairs contain no instances of blocking or forcing when n > 5. Neither do the options for the n queens problem when n > 3. There’s no “excess fat” in those specifications. In MacMahon’s triangle problem (exercise 126), Algorithm P needs just 20 megamems to remove 576 of the 1537 options; but the options that it removes don’t really matter, because Algorithm C traverses exactly the same search tree, with or without them.

We do gain 10% when we try to pack pentominoes into a 6 × 10 box (exercise 271): Without preprocessing, Algorithm X needs 4.11 Gμ to discover all 2339 solutions to that classic task. But Algorithm P needs just 0.19 Gμ to remove 235 of the 2032 options, after which Algorithm X finds the same 2339 solutions in 3.52 Gμ; so the total time has been reduced to 3.71 Gμ. The similar problem of packing the one-sided pentominoes into a 6 × 15 box has an even bigger payoff: It has 3308 options without preprocessing, and 15.5 Tμ are needed to process them. But after preprocessing — which costs a mere 260 Mμ — there are 3157 options, and the running time has decreased to 13.1 Tμ.

The simplifications discovered by Algorithm P for those pentomino problems involve only blocking (see exercise 225). But more subtle reductions occur in the Y pentomino problem of Fig. 73. For example, cell 20 is forced by cell 10, in that problem; and in round 2, cell 00 is forced by cell 22. In round 4, cell 61 is blocked by the option ‘50 51 52 53 62’ — a surprising discovery! Unfortunately, however, those clever reductions have little effect on the overall running time.

Preprocessing really shines on the problem of exercise 114, which asks for all sudoku solutions that are self-equivalent when reflected about their main diagonal. In this case Algorithm P is presented with 5410 options that involve intricate color controls, on 585 primary items and 90 secondary items. It rapidly reduces them to just 2426 options, on 506 primaries and 90 secondaries; and Algorithm C needs only 287 Gμ to process the reduced options and to discover the 30,286,432 solutions. That’s 7.5 times faster than the 2162 Gμ it would have needed without reduction.

Thus, preprocessing is a mixed bag. It might win big, or it might be a waste of time. We can hedge our bets by allocating a fixed budget — for instance by deciding that Algorithm P will be allowed to run at most a minute or so. Its data structures are in a “safe” state at the beginning of step P3; therefore we can jump from there directly to step P10 if we don’t want to run to completion.

Of course, preprocessing can also be applied to the subproblems that arise in the midst of a longer computation. A careful balancing of different strategies might be the key to solving problems that are especially tough.

Minimum-cost solutions

Many of the exact cover problems that we’ve been studying have few solutions, if any. In such cases our joy is to discover the rare gems. But in many other cases the problems have solutions galore; and for such problems we’ve focused our attention so far on the task of minimizing the amount of time per solution, assuming that all of the solutions are interesting.

A new perspective arises when each option of our problem has been assigned a nonnegative cost. Then it becomes natural to seek solutions of minimum cost. And ideally we’d like to do this without examining very many of the high-cost solutions at all; they’re basically useless, but a low-cost solution might be priceless.

Fortunately there’s a reasonably simple way to modify our algorithms, so that they will indeed find minimum-cost solutions rather quickly. But before we look at the details of those modifications, it will be helpful to look at several examples of what is possible.

Consider, for instance, the problem of Langford pairs from this point of view. We observed near the very beginning of Chapter 7 that there are 2L_16 = 653,443,600 ways to place the 32 numbers {1, 1, 2, 2, ..., 16, 16} into an array a_1a_2 ... a_32 so that exactly i entries lie between the two occurrences of i, for 1 ≤ i ≤ 16. And we claimed that the pairing displayed in 7–(3), namely

2 3 4 2 1 3 1 4 16 13 15 5 14 7 9 6 11 5 12 10 8 7 6 13 9 16 15 14 11 8 10 12,(106)

is one of 12,016 solutions that maximize the sum Σ1=Σk=132kak. Consequently the reverse of that pairing, namely

12 10 8 11 14 15 16 9 13 6 7 8 10 12 5 11 6 9 7 14 5 15 13 16 4 1 3 1 2 4 3 2,(107)

is one of 12,016 solutions that minimize ∑1. We noted in (16) above that Langford pairs are the solutions to a simple exact cover problem, whose options ‘i sj sk’ represent the assignments aj = i and ak = i. Therefore, if we associate the cost $(ji + ki) with option ‘i sj sk’, the minimum-cost solutions will be precisely the Langford pairings that minimize ∑1. (See exercise 226.)

One way to minimize the total cost is, of course, to visit all solutions and to compute the individual sums. But there’s a better way: The min-cost variant of Algorithm X below, which we shall call Algorithm X$, finds a solution of cost $3708 and proves its minimality after only 60 gigamems of computation. That’s more than 36 times faster than the use of plain vanilla Algorithm X, which needs 2.2 teramems to run through the full set of solutions.

Moreover, Algorithm X$ doesn’t stop there. It actually will compute the K solutions of least cost, for any given value of K. For example, if we take K = 12500, it needs just 70 gigamems to discover the 12,016 solutions of cost $3708, together with 484 solutions of the next-lowest cost (which happens to be $3720).

The news is even better when we try to minimize Σ2=Σk=132 k2ak instead of ∑1. Algorithm X$ needs just 28 Gμ to prove that the minimum ∑2 is $68880. And better yet is the fact, obtained in only 10 Gμ, that the minimum of Σk=132 kak2 is $37552, obtainable uniquely by the remarkable pairing

16 14 15 9 6 13 5 7 12 10 11 6 5 9 8 7 14 16 15 13 10 12 11 8 4 1 3 1 2 4 3 2,(108)

which also happens to minimize both ∑1 and ∑2! (See exercise 229.)

Another classic combinatorial task, the 16 queens problem, provides another instructive example. We know from previous discussions that there are exactly 14,772,512 ways to place 16 nonattacking queens on a 16 × 16 board. We also know that Algorithm X needs about 40 Gμ of computation to visit them all, when we give it options like (23).

Let’s suppose that the cost of placing a queen in cell (i, j) is the distance from that cell to the center of the board. (If we number the rows and columns from 1 to 16, that distance d(i, j) is (i−17/2)2+(j−17/2)2; it varies from d(8,8)=1/2  to  d(1,1)=15/2.) Thus we want to concentrate the queens near the center as much as possible, although many of them must lie at or near the edges because there must be one queen in each row and one queen in each column.

Fig. 74. Optimum solutions to the 16 queens problem, placing them (a) as close as possible to the center, or (b, c) as far as possible from it.

Figure 74(a) shows how to minimize the total cost — and this answer actually turns out to be unique, except for rotation and reflection. Similarly, Figs. 74(b) and 74(c) show the placements that maximize the cost. (Curiously, those solutions are obtainable from each other by reflecting the middle eight rows, without changing the top four or the bottom four.) Algorithm X$, with K = 9, discovers and proves the optimality of those placements in just (a) 3.7 Gμ; (b, c) 0.8 Gμ.

The modifications that convert Algorithm X to Algorithm X$ also convert Algorithm C into Algorithm C$. Therefore we can find minimum-cost solutions to XCC problems, which go well beyond ordinary exact cover problems.

For example, here’s a toy problem that now becomes tractable: Put ten different 5-digit prime numbers into the rows and columns of a 5 × 5 array, in such a way that their product is as small as possible. (A 5-digit prime number is one of the 8363 primes between 10007 and 99991, inclusive.) One such “prime square,” made up entirely of primes that are less than 30000, is

[2121120101110031106911113].(109)

To set this up as an XCC problem, introduce ten primary items {a_1, a_2, a_3, a_4, a_5} and {d_1, d_2, d_3, d_4, d_5} that represent “across” and “down,” together with 25 secondary items ij for 1 ≤ i, j ≤ 5 that represent cells of the array, together with 8363 additional secondary items p_1p_2p_3p_4p_5, one for eligible prime p = p_1p_2p_3p_4p_5. The options for placing p in row i or column j are then

‘aii1:p1i2:p2i3:p3i4:p4i5:p5p1p2p3p4p5’;‘dj1j:p12j:p23j:p34j:p45j:p5p1p2p3p4p5’.(110)

For example, ‘a_4 41:1 42:1 43:0 44:6 45:9 11069’ enables the prime 11069 in (109).

This is a good example where preprocessing is helpful, because the primes that are usable in a_1 and d_1 must not contain a 0; furthermore, the primes that are usable in a_5 and d_5 must contain only the digits {1, 3, 7, 9}. Algorithm P discovers those facts on its own, without being told anything special about number theory. It reduces the 83630 options of (110) to only 62900; and those reductions provide useful clues for the choices of items on which to branch.

The Monte Carlo estimate of exercise 86 tells us that there are roughly 6 × 10^14 different ways to fit ten primes into a 5 × 5 array — a vast number. We probably don’t need to look at too many of those possibilities, yet it isn’t easy to decide which of them can safely be left unexamined.

To minimize the product of the primes, we assign the cost $(ln p) to each of the options in (110). (This works because the logarithm of a product is the sum of the logarithms of the factors.) More precisely, we use the cost $ ⌊C ln p⌋, where C is large enough to make truncation errors negligible, but not large enough to cause arithmetic overflow, because Algorithm C$ wants all costs to be integers.

Every solution has the same cost as its transpose. Thus we can get the best five prime squares by asking Algorithm C$ to compute the K = 10 least-cost solutions, each of which occurs twice. Here they are, with the best at the left:

[1111310103110033176911171][1111310103110033177111197][1111310007110033169911117][1111310007110033166311177][1111310007110033166311197].(111)

The running time, 440 Gμ, would have been 1270 Gμ without preprocessing; so the 280 Gμ spent in preprocessing paid off. But the five greatest-cost solutions,

[9998988997986899979399991][9998989899967999873799991][9998988997988979957199971][9998988997988979958199991][9998988997988979957799971](112)

(greatest at the right), can be found in just 22 Gμ, without preprocessing.

Let’s turn now from purely mathematical problems to some “organic” scenarios that are more typical of the real world. The USA’s 48 contiguous states

define an interesting planar graph that has already supplied us with a variety of instructive examples. This graph G has 48 vertices and 105 edges. Suppose we want to partition it into eight connected subgraphs of six vertices each. What’s the minimum number of edges whose removal will do that?

Well, exercise 7.2.2–76 has told us how to list all of the connected subsets of six states, and there happen to be 11505 of them. That gives us 11505 options for an exact cover problem on 48 items, whose solutions are precisely the potential partitions of interest. The total number of solutions turns out to be 4,536,539; and Algorithm X is able to visit them all, at a cost of 807 gigamems.

But let’s try to do better, using Algorithm X$. Every induced subgraph G|U has an exterior cost, which is the number of edges from U to vertices not in U. When we partition a graph by removing edges, every such edge contributes to the exterior cost of two of the components that remain; hence the number of removed edges is exactly half the sum of the exterior costs. The best partition therefore corresponds to the minimum-cost solution to our exact cover problem, if we assign the exterior costs to each option. For example, one of the 11505 options is

‘ND SD NE KS OK TX’,(114)

and we assign a cost of $19 to that option.

Algorithm X$ now obligingly finds, in just 3.2 gigamems, that the optimum solution costs $72. Hence we get the desired partition by removing only 36 edges.

Before we look at the answer, let’s stare at the problem a bit longer, because we still haven’t discovered the best way to solve it! A closer examination shows that option (114) is useless, because it could never actually appear in any solution: It cuts the graph into two pieces, with 11 states to the left and 31 states to the right. (We encountered a similar situation earlier in (38).) In fact, 4961 of the 11505 options turn out to be unusable, for essentially the same reason. The state of Maine (ME), for example, belongs to 25 connected subgraphs of order 6; but we can easily see that the only way to get ME into the final partition is to group it with the other five states of New England (NH, VT, MA, CT, RI). Exercise 242 explains how to detect and reject the useless options quickly.

The remaining problem, which has 6544 options, is solved by Algorithm X in 327 Gμ and by Algorithm X$ in just 1046 Mμ.

Essentially the same methods will partition the graph nicely into six connected clusters of order eight. This time the exact cover problem has 40520 options after reduction, and a total of 4,177,616 solutions. But Algorithm X$ needs less than 2 Gμ to determine the minimum cost, which is $54.

Here are examples of the optimum partitions found, 8 × 6 and 6 × 8:

In each case the optimum can actually be achieved in two ways: On the left, one could swap the affiliations of VA and WV; on the right, a more complicated cyclic shuffle (MI NE LA VA) could be used.

It’s also instructive to solve a different kind of problem, namely to use census data and to partition G based on the population of each state. For example, let’s try to find eight connected clusters that each contain nearly the same number of people. The total population, P , of the 48 states was officially 306084180 in 2010. We want each cluster to represent P/8 people, or as close to P/8 as we can get. That’s about 38 million people per cluster.

The algorithm of exercise 242 will find and reduce all connected subgraphs whose total population x satisfies L ≤ x < U, for any given bounds L and U. If we take L > ⌊P/9⌋ (which is about 34 million) and U ≤ ⌈P/7⌉ (which is about 44 million), those candidate subgraphs will define an exact cover problem for which every solution uses exactly eight options, because 9x > P and 7x < P .

That algorithm proves that G contains 1,926,811 connected sets of states whose population lies in [34009354 .. 43726312); and it prunes away 1,571,057 of them, leaving 355,754. But that’s overkill. This problem has enough flexibility that its final solution can be expected to contain only sets whose population is quite close to 38 million. Therefore we might as well restrict ourselves to the range [37000000 .. 39000000) instead. There are 34,111 such options; surely they should be enough to solve our problem.

Well, that’s very plausible, but unfortunately it doesn’t work: Those 34,111 options have no solution, because Algorithm X can’t use them to cover NY (New York)! Notice that NY is an articulation point of G. The population of New York is about 19.4 million, and the combined population of the six New England states is about 14.3 million. Whatever option covers New York had better cover all of New England too, otherwise New England is stranded. So that makes 19.4 + 14.3 = 33.7 million people. New York’s only other neighbors are New Jersey (8.8 million) and Pennsylvania (12.7 million); adding either of them will put us over 42 million.

So we’re clearly not going to be able to cover New York with a cluster that’s close to the desired 38 million. We’ll either need a lightweight one (New York plus New England) or a heavyweight one (with New Jersey too). Let’s throw those two options in with the other 34,111.

Notice that this problem is quite different from the others we’ve been discussing, because its options vary greatly in size. One of the options contains just one state, CA (California), whose population is the largest (37.3 million); others contain up to fifteen states, almost spanning the continent from DE to NV.

Now we assign the cost $(x^2) to each option with population x, because the minimum-cost solutions will then minimize the squared deviations (x_1 − P/8)2 + ··· +(x_8 − P/8)2. (See exercise 243.) This works well; and Algorithm X$ needs only 3.3 gigamems to find the optimum solution below. The seven options not involving New York all contain between 37.3 and 38.1 million people.

A similar analysis, partitioning into six equipopulated clusters instead of eight, gives in 1.1 Gμ a minimum-cost solution whose six populations are all in the range [50650000 .. 51150000]. Both solutions are illustrated here, with the area of each vertex proportional to its population:

In both cases the solution is unique. (And in both cases, let’s face it, the solution is also pretty weird. Partitions like this could only be concocted by a computer. Exercise 246 discusses approximate solutions that are less eccentric.)

*Implementing the min-cost cutoffs

OK, we’ve now seen lots of reasons why Algorithms X$ and C$ are desirable. But how exactly can we obtain those algorithms by extending Algorithm X and C? It will suffice to describe Algorithm C$.

The mission of Algorithm C$ is to find the K min-cost solutions. More precisely, it should discover K solutions whose total cost is as small as possible, with the understanding that different solutions might have the same cost. Let’s imagine, for example, a problem that has exactly ten solutions, and that their costs are $3, $1, $4, $1, $5, $9, $2, $6, $5, $3, in the order that Algorithm C would discover them. Algorithm C$ won’t differ from Algorithm C until it has found K solutions, because those K might turn out to be the best. After K are known, however, it will be harder to please: It will accept a new solution only if that solution is better than one of the best K it knows. Thus if K = 3, say, the accepted solutions will have costs $3, $1, $4, $1, $2; Algorithm C$ won’t find the other five.

To implement that behavior, we maintain a BEST table, which contains the K least costs known so far. That table is “heap ordered,” with

BEST[⌊j/2⌋]≥BEST[j]for 1≤⌊j/2⌋<j≤K(117)

(see Eq. 5.2.3–(3)). In particular, BEST[1] will be the greatest of the least K costs, and we call it the cutoff value, T . Algorithm C$ will reject any solution whose cost is T or more. Initially, BEST[j] = ∞ for 1 ≤ j ≤ K; then every new solution of cost c < T will be “sifted” into the BEST table as in Algorithm 5.2.3H. The successive cutoff values in the example above, if K = 3, would therefore be ∞, ∞, 4, 3, 3, 3, 2, 2, 2, 2. If K = 4 they’d be ∞, ∞, ∞, 4, 4, 4, 3, 3, 3, 3.

Algorithm C$ adds a COST field to every node, thereby making each node 64 bits larger than before. Step C1$ stores the cost of every option, assumed to be a nonnegative integer, in each node belonging to that option.

The costs in every list of options created by step C1$ are ordered, so that

COST(x)≤COST(y)if y=DLINK(x),(118)

whenever neither x nor y is a header node. Therefore, if p is primary and belongs to t options, we have COST(x_1) ≤ COST(x_2) ≤ ··· ≤ COST(xt), where x_1 = DLINK(p), xj+1 = DLINK(xj) for 1 ≤ j < t, and p = DLINK(xt). This fact will allow us to ignore options that are too expensive to be part of a min-cost solution.

For this purpose we generalize the basic operations of covering, purifying, uncovering, and unpurifying (see (50)–(57)), by including a threshold parameter ϑ: Their loops in Algorithm C$ now say ‘While q ≠ i and COST(q) < ϑ’ instead of simply ‘While q ≠ i’. We also change the uncovering and unpurifying operations, so that they now go downward using DLINKs instead of upward using ULINKs. Furthermore, we make the unhiding operation of (15) go from left to right, with q increasing, just as hiding does in (13). (These conventions clearly flout the rules by which we established the validity of dancing links in the first place! But we’re lucky, because they’re justified by the theory in exercise 2.)

At level l of the search, Algorithm C$ has constructed a partial solution, consisting of l options represented by nodes x_0 ... xl−1. Let Cl be their total cost. In step C4$ we set xl ← DLINK(i), then cover item i using the threshold value ϑ_0 = T −Cl −COST(xl). (Item i will have been chosen so that ϑ_0 > 0.) The covering process will now proceed faster than before, if ϑ_0 is fairly low, because it won’t bother to hide options that could not be in an accepted solution. We need to remember the value of ϑ_0, so that exactly the same threshold will be used when backtracking; therefore step C4$ sets TH0[l] ← ϑ_0, and step C7$ uses TH0[l] as the threshold for uncovering item i, where TH0 is an auxiliary array.

The cutoff value T decreases as computation proceeds. Therefore the threshold ϑ = T − Cl − COST(xl) used in step C5$ for covering and purification might be different each time. Step C5$ should go directly to C7$ if ϑ ≤ 0. Otherwise it sets TH[l] ← ϑ in that step, and uses TH[l] for undoing in step C6$, where TH is another auxiliary array.

Step C3$, which chooses the item on which to branch at level l, is of course crucially important. If some primary item i has no options, or if the cost of its least expensive option is so high that it can’t lead to a solution better than we’ve already found, step C3$ should jump immediately to step C8$. Otherwise, many strategies are worthy of investigation, and there’s room here to discuss only the method that was used in the author’s experiments: Good results were obtained by choosing an i with the fewest not-too-costly options, as in the MRV heuristic. In case of ties, the author’s implementation chose an i whose least expensive option cost the most. (That item must be covered sooner or later, so there’s no way to avoid paying that much. We probably have a better chance of reaching a cutoff quickly if we maximize our chances of failure.) Exercise 248 has full details.

Many applications of Algorithm C$ have special features that allow us to prune unproductive branches from the search tree long before they would be cut off by the methods discussed so far. For example, every option in our “square of primes” problem has exactly one primary item (see (110)). In such cases, we know that every solution obtained by extending x_0 ... xl−1 must cost at least

Cl+COST(DLINK(i1))+...+COST(DLINK(it)),(119)

because of (118), where i_1, ..., it are the primary items still active. If this total is T or more, step C3$ can proceed immediately to step C8$.

Similarly, in the n queens problem, every option has exactly two primary items, one of the form Ri and one of the form Cj. The active items i_1, ..., it must therefore contain t/2 of each. Let CR and CC be Σ COST(DLINK(ij)), summed over those types. If either Cl + CR or Cl + CC is ≥ T , step C3$ can jump to C8$.

In our first problem for the contiguous USA, every option has exactly 6 items. (See (114).) Hence the number of active items, t, is always a multiple of 6. Exercise 249 presents a nice algorithm to find the least possible cost of t/6 future options; step C3$ of Algorithm C$ uses that method to find early cutoffs.

The Langford pair problem has options with three items, one of which is a digit. The pentomino problems have options with six items, one of which is a piece name. In both cases, Algorithm C$ can obtain suitable lower bounds for early cutoffs by combining the strategies already mentioned. (See exercise 250.)

Finally, Algorithm C$ uses one other important technique: It gains traction by preprocessing the costs. Notice that if p is a primary item, and if the cost of every option that includes p is c or more, we could decrease the cost of all those options by c, without changing the set of min-cost solutions. That’s true because p is going to appear exactly once in every solution. We can think of c as an unavoidable tax or “cover charge,” which must be paid “up front.”

In general there are many ways to preprocess the costs without changing the underlying problem. Properly transformed costs can help the algorithm’s heuristics to make much more intelligent choices. Exercise 247 discusses the simple method that was used for step C1$ in the author’s experiments discussed earlier.

*Dancing with ZDDs

The solutions visited by Algorithm X in step X2 can be represented naturally in the form of a decision tree, as we discussed in Section 7.1.4. For example, here’s a decision tree for the solutions to the problem of covering the eight cells of a 3 × 3 board with four dominoes, after the corner cell 22 has been removed:

This diagram uses the standard ZDD conventions: Every branch node names an option. A solid line means that the option is taken, while a broken line means that it is not. The terminal nodes ⊥ and ⊤ represent failure and success. This problem has four solutions, corresponding to the four paths from the root to ⊤.

We learned in Section 7.1.4 that ZDDs can readily be manipulated, and that a small ZDD can sometimes characterize a large family of solutions. If we’re lucky, we can save a huge amount of time and energy by simply generating an appropriate ZDD instead of visiting the solutions one by one.

Such economies arise when the same subproblem occurs repeatedly. For example, two branches come together in (120) at the node marked ‘*’; this happens because the problem that remains after placing two dominoes ‘00 10’ and ‘01 11’ is the same as the residual problem after placing ‘00 01’ and ‘10 11’. “We’ve been there and done that.” Hence we needn’t recapitulate our former actions, if we’ve already built a subZDD to remember what we did. (Those two pairs of domino placements form a “bipair,” as discussed earlier; but the ZDD idea is considerably more general and powerful.)

Let’s look more closely at the underlying details. The exact cover problem solved by (120) has eight items 00, 01, 10, 02, 11, 20, 12, 21, representing cells to be covered; and it has the following ten options, representing domino placements:

1:00 013:01 025:02 127:10 209:11 212:00 104:01 116:10 118:11 1210:20 21(121)

The ZDD (120) is internally represented as a sequence of branching instructions,

I12=(2¯?8:11),I11=(4¯?10:3),I10=(3¯?0:9),I9=(8¯?0:2),I8=(1¯?0:7),I7=(7¯?4:6),I6=(5¯?0:5),I5=(9¯?0:1),I4=(6¯?0:3),I3=(5¯?0:2),I2=(10¯?0:1),(122)

where 0 and 1 stand for ⊥ and ⊤. (See, for instance, 7.1.4–(8).) “If we don’t take option 2, go to instruction 8; but if we do take it, continue with instruction 11.”

A few modifications to Algorithm X will transform it from a solution-visiting method into a constructor of ZDDs. In fact, color controls can be handled too:

Algorithm Z (Dancing with ZDDs). Given an XCC problem as in Algorithm C, this algorithm outputs a free ZDD for the sets of options that satisfy it. The ZDD instructions {I_2, ..., Is} have the form (o¯j?lj:hj) illustrated in (122), and Is is the root. (But if the problem has no solutions, the algorithm terminates with s = 1, and the root is 0.) The data structures of Algorithm C are extended by a “memo cache” consisting of signatures S[j] and ZDD pointers Z[j]. Algorithm C’s table of choices x_0x_1 ... is joined by two new auxiliary tables m_0m_1 ... and z_0z_1 ... , indexed by the current level l.

Z1. [Initialize.] Set the problem up in memory, as in step C1 of Algorithm C. Also set N to the number of items, Z to the last spacer address, l ← 0, S[0] ← 0, Z[0] ← 1, m ← 1, s ← 1.

Z2. [Enter level l.] Form a “signature” σ that characterizes the current subproblem (see below). If σ = S[t] for some t (this is a “cache hit”), set ζ ← Z[t] and go to Z8. Otherwise set S[m] ← σ, ml ← m, zl ← 0, and m ← m + 1.

Z3. [Choose i.] At this point items i_1, ..., it still need to be covered, as in step C3 of Algorithm C. Choose one of them, and call it i.

Z4. [Cover i.] Cover item i using (12), and set xl ← DLINK(i).

Z5. [Try xl.] If xl = i, go to Z7. Otherwise set p ← xl + 1, and do the following while p ≠ xl: Set j ← TOP(p); if j ≤ 0, set p ← ULINK(p); otherwise commit(p, j) and set p ← p + 1. Set l ← l + 1 and return to Z2.

Z6. [Try again.] Set p ← xl − 1, and do the following while p ≠ xl: Set j ← TOP(p); if j ≤ 0, set o ← 1 − j and p ← DLINK(p); otherwise uncommit(p, j) and set p ← p − 1. If ζ ≠ 0, set s ← s + 1, output Is=(o¯?zl:ζ), and set zl ← s. Set i ← TOP(xl), xl ← DLINK(xl), and return to Z5.

Z7. [Backtrack.] Uncover item i using (14). Then set Z[ml] ← zl and ζ ← zl.

Z8. [Leave level l.] Terminate if l = 0. Otherwise set l ← l − 1 and go to Z6.

Important: The ‘commit’ and ‘uncommit’ operations in steps Z5 and Z6 should modify (54)–(57), by calling cover(j), hide(q), uncover(j), and unhide(q) instead of cover′(j), hide′(q), uncover′(j), and unhide′(q). These changes cause every step of Algorithm Z to be slightly different from the corresponding step of Algorithm C. (Yet only step Z2 has changed substantially.)

Exercise 253 shows that a few more changes will make Algorithm Z compute the total number of solutions, instead of (or in addition to) outputting a ZDD.

The keys to Algorithm Z’s success are the signatures computed in step Z2. This computation is easy if there are no secondary items: The signature σ is then simply a bit vector of length N, containing 1 in every position i where item i is still active. The computation is, however, somewhat subtler in the presence of secondary items; exercise 254 has the details.

It’s instructive to analyze some special cases. For example, suppose Algorithm Z is asked to find the perfect matchings of the complete graph KN . This problem has N primary items {1, ..., N}, and (N2) options ‘j k’ for 1 ≤ j < k ≤ N. We noted earlier, in the discussion preceding (87), that every item list on level l has exactly N − 1 − 2l options, regardless of the choices made in step Z3. If we always choose the smallest uncovered item, step Z2 will compute exactly (N−ll) different signatures on level l, namely the signatures in which items {1, ..., l} are covered and so are l of the other items {l + 1, ..., N}. Hence the total number of cache entries is Σl=0N(N−ll)=FN+1, a Fibonacci number(!). (See exercise 1.2.8–16.) Moreover, the main loops in steps Z5 and Z6 are executed (N−1−2l)(N−ll) times at level l, since steps Z3 and Z4 are executed (N−ll) times.

In fact, when N is even, the ZDD that is output for all perfect matchings of KN turns out to have exactly Σl=0N(N−1−2l)(N−ll)+2 nodes, which is approximately N5FN+1. Exercise 255 shows that the total running time to compute this ZDD is Θ(N^2FN) = Θ(N^2ϕN); and the same estimate holds also when N is odd and the ZDD has only one node ‘⊥’. This is much smaller than the time needed by Algorithm X, which is Θ ((N/e)N/2).

More concretely, Algorithm X computes the 2,027,025 perfect matchings of K_16 in about 360 megamems, using about 6 kilobytes of memory. Algorithm Z needs only about 2 megamems to characterize those matchings with a 10,228-node ZDD; but it uses 2.5 megabytes of memory. For K_32 there are 191,898, 783,962,510,625 perfect matchings, and the difference is even more dramatic: Algorithm X costs about 34 thousand petamems and 25 kilobytes; Algorithm Z costs about 16 gigamems and 85 megabytes, for a ZDD with 48 meganodes.

This example illustrates several important points: (1) Algorithm Z can greatly reduce the running time of Algorithm X (or Algorithm C), trading time for space. (2) These improvements can also be achieved for problems that have no solutions, like matchings of K_2q+1. (3) The number of nodes in the ZDD that is output might greatly exceed the number of memos in Algorithm Z’s cache.

Let’s take a closer look at the ZDDs. The output for N = 8 is, schematically,

where, for example, ‘00101101’ represents nodes for the signature 00101101.

A signature represents a subproblem. If that subproblem has at least one solution, the ZDD for the full problem will have a subZDD that specifies all solutions of the subproblem. And if the signature is in cache location S[t], the root of the corresponding subZDD will be stored in Z[t], at the end of step Z7.

This subZDD has a very special structure, illustrated in (123). Suppose we branch on item i when working on signature σ; and suppose solutions are found for options o_1, o_2, ..., ok in the list for item i. Then there will be subZDDs rooted at ζ_1, ζ_2, ..., ζk, associated with the subproblems whose signatures are σ \ o_1, σ \ o_2, ..., σ \ ok. The net effect of steps Z3–Z6 is to construct a subZDD for σ that essentially begins with k conditional instructions:

ok?ζk:...o2?ζ2:o1?ζ1:⊥.(124)

(The ZDD is constructed from bottom to top, so it tests ok first.)

For example, ‘00101101’ in (123) is the root of the subZDD for the subproblem that needs to cover {3, 5, 6, 8}. We branch on item 3, whose list has the three options ‘3 5’, ‘3 6’, ‘3 8’. Three branch instructions are output,

Iθ=(3  5¯?0:γ ),Iη=(3  6¯?θ:β ),Iζ=(3  8¯?η:α );
here γ, β, α are the subZDDs for signatures 00000101, 00001001, 00001100, respectively. The subZDD for 00101101 begins at ζ.

Thus (123) illustrates a ZDD with 1 · 7 + 7 · 5 + 15 · 3 + 10 · 1 + 2 = 99 nodes. Notice that the dotted links always go either to ⊥ or to an “invisible” node, which is one of the k − 1 subsidiary nodes in a chain of branches such as (124). Every invisible node has a single parent. But there is one visible node for each successful signature, and a visible node may have many parents.

Exercises 256–262 discuss a number of examples where Algorithm Z gives spectacular improvements over Algorithm X (and over Algorithm C when colors are involved). Many additional examples could also be given. But most of the exact cover problems we’ve been considering in our examples do not have an abundance of common subproblems, so they reap little benefit from the memo cache.

For example, consider our old standby, the problem of Langford pairs. Algorithm X needs 15 Gμ to show that there are no solutions when n = 14; Algorithm Z reduces this slightly, to 11 Gμ. Algorithm X needs 1153 Gμ to list the 326,721,800 solutions for n = 16; Algorithm Z computes that number in 450 Gμ; but it needs 20 gigabytes of memory, and produces a ZDD of 500 million nodes!

Similarly, Algorithm Z is not the method of choice for the n queens problem, or for word-packing problems, although it does yield modest speedups more often than one might suspect. Exercise 263 surveys some typical examples.

Summary

We began this section by observing that simple properties of linked lists can enhance the efficiency of backtracking, especially when applied to exact cover problems (XC). Then we noticed that a wide variety of combinatorial tasks, going well beyond matching, turn out to be special cases of exact covering.

The most important “takeaway,” however, has been the fact that color codes lead to a significant generalization of the classical exact cover problem. Indeed, the general XCC problem, “exact covering with colors,” has a truly extraordinary number of applications. The exercises below exhibit dozens and dozens of instructive problems that are quite naturally describable in terms of “options,” which involve “items” that may or may not be colored in certain ways. We’ve discussed Algorithm X (for XC problems) and Algorithm C (for XCC problems); and the good news is that those algorithms are almost identical.

Furthermore, we’ve seen how to extend Algorithm C in several directions: Algorithm M handles the general MCC problem, which allows items to be covered with different ranges of multiplicities. Algorithm C$ associates a cost with each option, and finds XCC solutions of minimum total cost. Algorithm Z produces XCC solutions as ZDDs, which can be manipulated and optimized in other ways.

Historical notes

The basic idea of (2) was introduced by H. Hitotumatu and K. Noshita [Information Processing Letters 8 (1979), 174–175], who applied it to the n queens problem. Algorithm 7.2.1.2X, which was published by J. S. Rohl in 1983, can be regarded as a simplified version of dancing links, for cases when singly linked lists suffice. (Indeed, as Rohl observed, the n queens problem is such a case.) Its extension to exact cover problems in general, as in Algorithm X above, was the subject of the author’s tribute to C. A. R. Hoare in Millennial Perspectives in Computer Science (2000), 187–214, where numerous examples were given. [That paper was subsequently reprinted with additions and corrections as Chapter 38 of FGbook.] His original implementation, called DLX, used a more complex data structure than (10), involving nodes with four-way links.

Knuth extended Algorithm X to Algorithm C in November 2000, while thinking about two-dimensional de Bruijn sequences. A special case of Algorithm M, in which all multiplicities are fixed, followed in August 2004, when he was thinking about packing various sizes of bricks into boxes. The current form of Algorithm M was developed in January 2017, after he’d studied an independent generalization of Algorithm X that Wei-Hwa Huang had written in 2007.

The first computer programs for exact cover problems were developed independently by J. F. Pierce [Management Science 15 (1968), 191–209] and by R. S. Garfinkel and G. L. Nemhauser [Operations Research 17 (1969), 848–856]. In both cases the given options each had an associated cost, and the goal was to obtain minimum-cost solutions instead of arbitrary solutions. Both algorithms were similar, although they used different ways to prune nonoptimum choices: Items were chosen for branching according to a fixed, precomputed order, and options were represented as bit vectors. An option was never removed from its item list; it would repeatedly be rejected if its bits intersected with previously chosen items. (Caution: Literature from the operations research community traditionally reverses the roles of rows and columns in matrices like (5). For them, items are rows and options are columns, even though bit vectors look like rows.)

The concept of “dancing with ZDDs” was introduced by M. Nishino, N. Yasuda, S. Minato, and M. Nagata, in the AAAI Conference on Artificial Intelligence 31 (2017), 868–874, where they presented the special case of Algorithm Z in which all items are primary.

The history of XCC solving is clearly still in its infancy, and much more work needs to be done. For example, many applications will benefit from improved ways to choose an item for branching — especially in step M3 of Algorithm M, where only a few strategies have been explored so far. It’s important to maintain a good “focus”; furthermore, techniques of “factoring” can dramatically prune away unproductive branches, as shown for example in exercise 343.

Algorithm M deserves to be extended to Algorithm M$, and perhaps also to produce ZDD output. A further generalization would be to allow each item of each option to have an associated weight. (Thus the associated matrix, analogous to (5), would not consist merely of 0s and 1s.)

Hence we can expect to see many continued advances in XCC solving.

Exercises—First Set

▸ 1. [M25] A doubly linked list of n elements, with a list head at 0, begins with LLINK(k) = k − 1 and RLINK(k − 1) = k for 1 ≤ k ≤ n; furthermore LLINK(0) = n and RLINK(n) = 0, as in (3). But after we use operation (1) to delete elements a_1, a_2, ..., an, where a_1a_2 ... an is a permutation of {1, 2, ..., n}, the list will be empty and the links will be entangled as in (4).

1. Show that the final settings of LLINK and RLINK can be described in terms of the binary search tree that is obtained when the keys an, ..., a_2, a_1 (in reverse order) are inserted by Algorithm 6.2.2T into an initially empty tree.

2. Say that permutations a_1a_2 ... an and b_1b_2 ... bn are equivalent if they both yield the same LLINK and RLINK values after deletion. How many distinct equivalence classes arise, for a given value of n?

3. How many of those equivalence classes contain just one permutation?

2. [M30] Continuing exercise 1, we know that the original list will be restored if we use (2) to undelete the elements an, ..., a_2, a_1, reversing the order of deletion.

1. Prove that it’s restored also if we use the unreversed order a_1, a_2, ..., an(!).

2. Is the original list restored if we undelete the elements in any order whatsoever?

3. What if we delete only k of the elements, say a_1, ..., ak, then undelete them in exactly the same order a_1, ..., ak. Is the list always restored?

3. [20] An m × n matrix that’s supposed to be exactly covered can be regarded as a set of n simultaneous equations in m unknowns. For example, (5) is equivalent to
x_2 + x_4 = x_3 + x_5 = x_1 + x_3 = x_2 + x_4 + x_6 = x_1 + x_6 = x_3 + x_4 = x_2 + x_5 + x_6 = 1,
where each xk = [choose row k] is either 0 or 1.

1. What is the general solution to those seven equations?

2. Why is this approach to exact cover problems almost never useful in practice?

4. [M20] Given a graph G, construct a matrix with one row for each vertex v and one column for each edge e, putting the value [e touches υ] into column e of row υ. What do the exact covers of this “incidence matrix” represent?

5. [18] Among the many combinatorial problems that can be formulated in terms of 0–1 matrices, some of the most important deal with families of sets: The columns of the matrix represent elements of a given universe, and the rows represent subsets of that universe. The exact cover problem is then to partition the universe into such subsets. In geometric contexts, an exact cover is often called a tiling.

Equivalently, we can use the terminology of hypergraphs, speaking of hyperedges (rows) that consist of vertices (columns); then the exact cover problem is to find a perfect matching, also called a perfect packing, namely a set of nonoverlapping hyperedges that hit every vertex.

Such problems generally have duals, which arise when we transpose the rows and columns of the input matrix. What is the dual of the exact cover problem, in hypergraph terminology?

6. [15] If an exact cover problem has N items and M options, and if the total length of all options is L, how many nodes are in the data structures used by Algorithm X?

7. [16] Why is TOP(23) = −4 in Table 1? Why is DLINK(23) = 25?

8. [22] Design an algorithm to set up the initial memory contents of an exact cover problem, as needed by Algorithm X and illustrated in Table 1. The input to your algorithm should consist of a sequence of lines with the following format:

- The very first line lists the names of all items.

- Each remaining line specifies the items of a particular option, one option per line.

9. [18] Explain how to branch in step X3 on an item i for which LEN(i) is minimum. If several items have that minimum length, i itself should also be minimum. (This choice is often called the “minimum remaining values” (MRV) heuristic.)

10. [20] In some applications the MRV heuristic of exercise 9 leads the search astray, because certain primary items have short lists yet convey little information about desirable choices. Modify answer 9 so that an item p whose name does not begin with the character ‘#’ will be chosen only if LEN(p) ≤ 1 or no other choices exist. (This tactic is called the “sharp preference” heuristic.)

▸ 11. [19] Play through Algorithm X by hand, using exercise 9 in step X3 and the input in Table 1, until first reaching step X7. What are the contents of memory at that time?

▸ 12. [21] Design an algorithm that prints the option associated with a given node x, cyclically ordering the option so that TOP(x) is its first item. Also print the position of that option in the vertical list for that item. (For example, if x = 21 in Table 1, your algorithm should print ‘d f a’ and state that it’s option 2 of 3 in the list for item d.)

13. [16] When Algorithm X finds a solution in step X2, how can we use the values of x_0x_1 ... xl−1 to figure out what that solution is?

▸ 14. [20] (Problème des ménages.) “In how many ways can n male-female couples sit at a circular table, with men and women alternating, and with no couples adjacent?”

1. Suppose the women have already been seated, and let the vacant seats be (S_0, S_1, ..., Sn−1). Let Mj be the spouse of the woman between seats Sj and S(j+1) mod n. Formulate the ménage problem as an exact cover problem with items Sj and Mj.

2. Apply Algorithm X to find the solutions for n ≤ 10. Approximately how many mems are needed per solution, with and without the MRV heuristic?

15. [20] The options in (16) give us every solution to the Langford pair problem twice, because the left-right reversal of any solution is also a solution. Show that, if a few of those options are removed, we’ll get only half as many solutions; the others will be the reversals of the solutions found.

16. [16] What are the solutions to the four queens problem, as formulated in (23) and (24)? What branches are taken at the top four levels of Algorithm X’s search tree?

17. [16] Repeat exercise 16, but consider aj and bj to be secondary items and omit the slack options (24). Consider the primary items in order r_3, c_3, r_2, c_2, r_4, c_4, r_1, c_1.

18. [10] What are the solutions to (6) if items e, f, and g are secondary?

▸ 19. [21] Modify Algorithm X so that it doesn’t require the presence of any primary items in the options. A valid solution should not contain any purely secondary options; but it must intersect every such option. (For example, if only items a and b of (6) were primary, the only valid solution would be to choose options ‘a d g’ and ‘b c f’.)

▸ 20. [25] Generalize (26) to a pairwise ordering of options (α_0, ..., αm−1; β_0, ..., βm−1) that uses at most ⌈lg m⌉ of the secondary items y_1, ..., ym−1 in each option. Hint: Think of binary notation, and use yj at most 2ρj times within each of the α’s and β’s.

21. [22] Extend exercise 20 to k-wise ordering of km options αji, for 1 ≤ i ≤ k and 0 ≤ j < m. The solutions should be (αj1i,...,αjkk) with 0 ≤ j_1 ≤ … ≤ jk < m. Again there should be at most ⌈lg m⌉ secondary items in each option.

▸ 22. [28] Most of the solutions to the n queens problem are unsymmetrical, hence they lead to seven other solutions when rotated and/or reflected. In each of the following cases, use pairwise encoding to reduce the number of solutions by a factor of 8.

1. No queen is in either diagonal, and n is odd.

2. Only one of the two diagonals contains a queen.

3. There are two queens in the two diagonals.

23. [28] Use pairwise encoding to reduce the number of solutions by nearly a factor of 8 in the remaining cases not covered by exercise 22:

1. No queen is in either diagonal, and n is even.

2. A queen is in the center of the board, and n is odd.

24. [20] With Algorithm X, find all solutions to the n queens problem that are unchanged when they’re rotated by (a) 180°; (b) 90°.

25. [20] By setting up an exact cover problem and solving it with Algorithm X, show that the queen graph Q_8 (exercise 7.1.4–241) cannot be colored with eight colors.

26. [21] In how many ways can the queen graph Q_8 be colored in a “balanced” fashion, using eight queens of color 0 and seven each of colors 1 to 8?

27. [22] Introduce secondary items cleverly into the options (16), so that only planar solutions to Langford’s problem are obtained. (See exercise 7–8.)

28. [M22] For what integers c_0, t_0, c_1, t_1, ..., cl, tl with 1 ≤ cj ≤ tj does the text’s formula (27) for estimated completion ratio give the value (a) 1/2? (b) 1/3?

▸ 29. [26] Let T be any tree. Construct the 0–1 matrix of an unsolvable exact cover problem for which T is the backtrack tree traversed by Algorithm X with the MRV heuristic. (A unique item should have the minimum LEN value whenever step X3 is encountered.) Illustrate your construction when

30. [25] Continuing exercise 29, let T be a tree in which certain leaves have been distinguished from the others and designated as “solutions.” Can all such trees arise as backtrack trees in Algorithm X?

31. [M21] The running time of Algorithm X depends on the order of primary items in the active list, as well as on the order of options in the individual item lists. Explain how to randomize the algorithm so that (a) every item list is in random order after step X1; (b) step X3 chooses randomly among items with the minimum LEN.

32. [M21] The solution to an exact cover problem with M options can be regarded as a binary vector x = x_1 ... xM , with xk = [choose option k]. The distance between two solutions x and x′ can then be defined as the Hamming distance d(x, x′) = ν(x⨁x′), the number of places where x and x′ differ. The diversity of the problem is the minimum distance between two of its solutions. (If there’s at most one solution, the diversity is ∞.)

1. Is it possible to have diversity 1?

2. Is it possible to have diversity 2?

3. Is it possible to have diversity 3?

4. Prove that the distance between solutions of a uniform exact cover problem — that is, a problem having the same number of items in each option — is always even.

5. Most of the exact cover problems that arise in applications are at least quasi-uniform, in the sense that they have a nonempty subset of primary items such that the problem is uniform when restricted to only those items. (For example, every polyomino or polycube packing problem is quasi-uniform, because every option specifies exactly one piece name.) Can such problems have odd distances?

33. [M16] Given an exact cover problem, specified by a 0–1 matrix A, construct an exact cover problem A′ that has exactly one more solution than A does. [Consequently it is NP-hard to determine whether an exact cover problem with at least one solution has more than one solution.] Assume that A contains no all-zero rows.

34. [M25] Given an exact cover problem A as in exercise 33, construct an exact cover problem A′ such that (i) A′ has at most three 1s in every column; (ii) A′ and A have exactly the same number of solutions.

35. [M21] Continuing exercise 34, construct A′ having exactly three 1s per column.

▸ 36. [25] Let ik = TOP(xk) be the item on which branching occurs at level k in Algorithm X. Modify that algorithm so that it finds the solution for which i_0x_0i_1x_1i_2x_2 ... is smallest in lexicographic order. (It’s easy to do this by simply setting i ← RLINK(0) in step X3. But there’s a much faster way, by using the MRV heuristic most of the time.)

What is the lexicographically first solution to the 32 queens problem?

37. [M46] (N. J. A. Sloane, 2016.) Let ⟨qn⟩ be the lexicographically smallest solution to the ∞ queens problem. (This sequence begins 1, 3, 5, 2, 4, 9, 11, 13, 15, 6, 8, 19, 7, 22, 10, 25, 27, 29, 31, 12, 14, 35, 37, 39, 41, 16, 18, 45, ..., and it clearly has strange regularities and irregularities.)

1. Prove that every positive integer occurs in the sequence.

2. Prove that qn is either nϕ + O(1) or n/ϕ + O(1).

▸ 38. [M25] Devise an efficient way to compute the sequence ⟨qn⟩ of exercise 37.

▸ 39. [M21] Experiment with exact cover problems that are defined by m random options on n items. (Each option is generated independently, with repetitions permitted.)

1. Use a fixed probability p that item i is included in any given option.

2. Let every option be a random sample of r distinct items.

▸ 40. [21] If we merely want to count the number of solutions to an exact cover problem, without actually constructing them, a completely different approach based on bitwise manipulation instead of list processing is sometimes useful.

The following naïve algorithm illustrates the idea: We’re given an m × n matrix of 0s and 1s, represented as n-bit vectors r_1, ..., rm. The algorithm works with a (potentially huge) database of pairs (sj, cj), where sj is an n-bit number representing a set of items, and cj is a positive integer representing the number of ways to cover that set exactly. Let p be the n-bit mask that represents the primary items.

N1. [Initialize.] Set N ← 1, s_1 ← 0, c_1 ← 1, k ← 1.

N2. [Done?] If k > m, terminate; the answer is Σj=1Ncj[sj & p=p].

N3. [Append rk where possible.] Set t ← rk. For N ≥ j ≥ 1, if sj & t = 0, insert (sj + t, cj) into the database (see below).

N4. [Loop on k.] Set k ← k + 1 and return to N2.

To insert (s, c) there are two cases: If s = si for some (si, ci) already present, we simply set ci ← ci + c. Otherwise we set N ← N + 1, sN ← s, cN ← c.

Show that this algorithm can be significantly improved by using the following trick: Set uk←rk & f¯k, where fk = rk+1 | ··· | rm is the bitwise OR of all future rows. If uk ≠ 0, we can remove any entry from the database for which sj does not contain uk & p. We can also exploit the nonprimary items of uk to compress the database further.

41. [25] Implement the improved algorithm of the previous exercise, and compare its running time to that of Algorithm X when applied to the n queens problem.

42. [M21] Explain how the method of exercise 40 could be extended to give representations of all solutions, instead of simply counting them.

43. [M20] Give formulas for the entries aij, bij, cij of the sudoku squares in (28).

44. [M04] Could the clues of a sudoku puzzle be the first 33 digits of π? (See (29a).)

45. [14] List the sequence of naked single moves by which Algorithm X cruises to the solution of (29a). (If several such pij are possible, choose the smallest ij at each step.)

46. [19] List all of the hidden single sudoku moves that are present in chart (31).

47. [19] What hidden singles are present in (32), after ‘3’ is placed in cell (2, 3)?

▸ 48. [24] Chart (33) essentially plots rows versus columns. Show that the same data could be plotted as either (a) rows versus values; or (b) values versus columns.

▸ 49. [24] Any solution to an exact cover problem will also solve the “relaxed” subproblems that are obtained by removing some of the items. For example, we might relax a sudoku problem (30) by removing all items cjk and bxk, as well as rik with i ≠ i_0. Then we’re left with a subproblem in which every option contains just two items, ‘pi_0 j ri_0 k’, for certain pairs (j, k). In other words, we’re left with a 2D matching problem.

Consider the bipartite graph with uj −−− υk whenever a sudoku option contains ‘pi_0 j ri_0 k’. For example, the graph for i_0 = 4 in (33) is illustrated below. A perfect matching of this graph must take u_3 and u_8 to either υ_7 or υ_1, hence the edges from other u’s to those υ’s can be deleted; that’s called a “naked pair” in row i_0. Dually, υ_5 and υ_8 must be matched to either u_2 or u_7, hence the edges from other υ’s to those u’s can be deleted; that’s called a “hidden pair” in row i_0.

In general, q of the u’s form a naked q-tuple if their neighbors include only q of the υ’s; and q of the υ’s form a hidden q-tuple if their neighbors include only q of the u’s.

1. These definitions have been given for rows. Show that naked and hidden q-tuples can be defined analogously for (i) columns, (ii) boxes.

2. Prove that if the bipartite graph has r vertices in each part, it has a hidden q-tuple if and only if it has a naked (r − q)-tuple.

3. Find all the naked and hidden q-tuples of (33). What options do they rule out?

4. Consider deleting items pij and bxk, as well as all rik and cjk for k ≠ k_0. Does this lead to further reductions of (33)?

50. [20] How many uniquely solvable 17-clue puzzles contain the 16 clues of (29c)?

51. [22] In how many ways can (29c) be completed so that every row, every column, and every box contains a permutation of the multiset {1, 2, 3, 4, 5, 6, 7, 7, 9}?

52. [40] Try to find a sudoku puzzle that’s as difficult as possible for Algorithm X.

53. [M26] Beginners to sudoku might want to cut their teeth on a miniature variant called shidoku, which features 4 × 4 squares divided into four 2 × 2 boxes.

1. Prove that every uniquely solvable shidoku problem has at least four clues.

2. Two shidoku problems are equivalent if we can get from one to the other by permuting rows and columns in such a way that boxes are preserved, and/or by 90° rotation, and/or by permuting the numbers. Show that exactly 13 essentially different 4-clue shidoku problems have a unique solution.

▸ 54. [35] (Minimal clues.) Puzzle (29a) contains more clues than necessary to make the sudoku solution unique. (For example, the final ‘95’ could be omitted.) Find all subsets X of those 32 clues for which (i) the solution is unique, given X; yet also (ii) for every x ∈ X, the solution is not unique, given X \ x.

55. [34] (G. McGuire.) Prove that at least 18 clues are necessary, in any sudoku puzzle whose unique answer is (28a). Also find 18 clues that suffice. Hint: At least two of the nine appearances of {1, 4, 7} in the top three rows must be among the clues.

Similarly, find a smallest-possible set of clues whose unique answer is (28b).

56. [47] What is the largest number of clues in a minimal sudoku puzzle?

57. [22] Every sudoku solution has at most 27 horizontal trios and 27 vertical trios, namely the 3-digit sets that appear within a single row or column of a box. For example, (28a) has nine horizontal trios {1, 2, 3}, {2, 3, 4}, ..., {9, 1, 2} and three vertical trios {1, 4, 7}, {2, 5, 8}, {3, 6, 9}; (28b) has just three of each. The solution to (29a) has 26 horizontal trios and 23 vertical trios; {3, 6, 8} occurs once horizontally, twice vertically.

Let T be the 27 trios {{A, B, C} | A ∈ {1, 2, 3}, B ∈ {4, 5, 6}, C ∈ {7, 8, 9}}. Find all sudoku solutions whose horizontal trios and vertical trios are both equal to T .

▸ 58. [22] (A. Thoen and A. van de Wetering, 2019.) Find all sudoku solutions for which the 1s, 2s, ..., 7s also solve the nine queens problem.

59. [20] Solve the jigsaw sudokus in (34). How large is Algorithm X’s search tree?

60. [20] (The Puzzlium Sudoku ABC.) Complete these hexomino-shaped boxes:

61. [21] Turn Behrens’s 5 × 5 gerechte design (35a) into a jigsaw sudoku puzzle, by erasing all but five of its 25 entries.

▸ 62. [34] For n ≤ 7, generate all of the ways in which an n×n square can be packed with n nonstraight n-ominoes. (These are the possible arrangements of boxes in a square jigsaw sudoku.) How many of them are symmetric? Hint: See exercise 7.2.2–76.

63. [29] In how many different ways can Behrens’s 9 × 9 array (35c) be regarded as a gerechte latin square? (In other words, how many decompositions of that square into nine boxes of size 9 have a complete “rainbow” {1, 2, 3, 4, 5, 6, 7, 8, 9} in each box? None of the boxes should simply be an entire row or an entire column.)

64. [23] (Clueless jigsaw sudoku.) A jigsaw sudoku puzzle can be called “clueless” if its solution is uniquely determined by the entries in a single row or column, because such clues merely assign names to the n individual symbols that appear. For example, the first such puzzle to be published, discovered in 2000 by Oriel Maxime, is shown here.

1. Find all clueless sudoku jigsaw puzzles of order n ≤ 6.

2. Prove that such puzzles exist of all orders n ≥ 4.

65. [24] Find the unique solutions to the following examples of jigsaw sudoku:

▸ 66. [30] Arrange the following sets of nine cards in a 3 × 3 array so that they define a sudoku problem with a unique solution. (Don’t rotate them.)

▸ 67. [22] Hypersudoku extends normal sudoku by adding four more (shaded) boxes in which a complete “rainbow” {1, 2, 3, 4, 5, 6, 7, 8, 9} is required to appear:

(Such puzzles, introduced by P. Ritmeester in 2005, are featured by many newspapers.)

1. Show that a hypersudoku solution actually has 18 rainbow boxes, not only 13.

2. Use that observation to solve hypersudoku puzzles efficiently by extending (30).

3. How much does that observation help when solving (i) and (ii)?

4. True or false: A hypersudoku solution remains a hypersudoku solution if the four 4 × 4 blocks that touch its four corners are simultaneously rotated 180°, while also flipping the middle half-rows and middle half-columns (keeping the center fixed).

68. [28] A polyomino is called convex if it contains all of the cells between any two of its cells that lie in the same row or the same column. (This happens if and only if it has the same perimeter as its minimum bounding box does, because each row and column contribute 2.) For example, all of the pentominoes (3^6) are convex, except for ‘U’.

1. Generate all ways to pack n convex n-ominoes into an n × n box, for n ≤ 7.

2. In how many ways can nine convex nonominoes be packed into a 9 × 9 box, when each of them is small enough to fit into a 4 × 4? (Consider also the symmetries.)

▸ 69. [30] Diagram (i) below shows the 81 communities of Bitland, and their nine electoral districts. The voters in each community are either Big-Endian (B) or Little-Endian (L). Each district has a representative in Bitland’s parliament, based on a majority vote.

Notice that there are five Ls and four Bs in every district, hence the parliament is 100% Little-Endian. Everybody agrees that this is unfair. So you have been hired as a computer consultant, to engineer the redistricting.

A rich bigwig secretly offers to pay you a truckload of money if you get the best possible deal for his side. You could gerrymander the districts as in diagram (ii), thereby obtaining seven Big-Endian seats. But that would be too blatantly biased.

Show that seven wins for B are actually obtainable with nine districts that do respect the local neighborhoods of Bitland quite decently, because each of them is a convex nonomino that fits in a 4 × 4 square (see exercise 68).

70. [21] Dominosa is a solitaire game in which you “shuffle” the 28 pieces , of double-six dominoes and place them at random into a 7 × 8 frame. Then you write down the number of spots in each cell, put the dominoes away, and try to reconstruct their positions based only on that 7 × 8 array of numbers. For example,

1. Show that another placement of dominoes also yields the same matrix of numbers.

2. What domino placement yields the array

(33651515656123242433362041614460303011442625050025054216)?

▸ 71. [20] Show that Dominosa reconstruction is a special case of 3D MATCHING.

72. [M22] Generate random instances of Dominosa, and estimate the probability of obtaining a 7 × 8 matrix with a unique solution. Use two models of randomness: (i) Each matrix whose elements are permutations of the multiset {8 × 0, 8 × 1, ..., 8 × 6} is equally likely; (ii) each matrix obtained from a random shuffle of the dominoes is equally likely.

73. [46] What’s the maximum number of solutions to an instance of Dominosa?

74. [22] (M. Keller, 1987.) Is there a uniquely solvable Dominosa array for which every domino matches two adjacent cells of the array in either three or four places?

▸ 75. [M24] A grope is a set G together with a binary operation ∘, in which the identity x ∘ (y ∘ x) = y is satisfied for all x ∈ G and y ∈ G.

1. Prove that the identity (x ∘ y) ∘ x = y also holds, in every grope.

2. Which of the following “multiplication tables” define a grope on {0, 1, 2, 3}?

0123103223013210;0321321021031032;0132102332102301;0231310213202031;0312213030211203.

(In the first example, x ∘ y = x ⊕ y; in the second, x ∘ y = (−x − y) mod 4. The last two have x ∘ y = x ⊕ f(x ⊕ y) for certain functions f.)

3. For all n, construct a grope whose elements are {0, 1, ..., n − 1}.

4. Consider the exact cover problem that has n^2 items xy for 0 ≤ x, y < n and the following n +(n^3 − n)/3 options:

1. ‘xx’, for 0 ≤ x < n;

2. ‘xx xy yx’, for 0 ≤ x < y < n;

3. ‘xy yz zx’, for 0 ≤ x < y,z < n.

Show that its solutions are in one-to-one correspondence with the multiplication tables of gropes on the elements {0, 1, ..., n − 1}.

5. Element x of a grope is idempotent if x ∘ x = x. If k elements are idempotent and n − k are not, prove that k ≡ n^2 (modulo 3).

76. [21] Modify the exact cover problem of exercise 75(d) in order to find the multiplication tables of (a) all idempotent gropes — gropes such that x ∘ x = x for all x; (b) all commutative gropes — gropes such that x ∘ y = y ∘ x for all x and y; (c) all gropes with the identity element 0 — gropes such that x ∘ 0 = 0 ∘ x = x for all x.

77. [M21] Given graphs G and H, each with n vertices, use Algorithm X to decide whether or not G is isomorphic to a subgraph of H. (In such a case we say that G is embedded in H.)

78. [16] Show that it’s quite easy to pack the 27 mathematicians’ names of Fig. 71 into a 12 × 15 array, with all names reading correctly from left to right. (Of course that would be a terrible word search puzzle.)

79. [M20] How many options are in (48), when they are completely listed?

80. [19] Play through Algorithm C by hand, using exercise 9 in step C3 and the input in Table 2, until first reaching a solution. What are the contents of memory then?

81. [21] True or false: An exact cover problem that has no color assignments has exactly the same running time on Algorithms X and C.

82. [21] True or false: It’s possible to save memory references in Algorithms X and C by not updating the LEN fields in the hide/unhide operations when x > N_1.

▸ 83. [20] Algorithm C can be extended in the following curious way: Let p be the primary item that is covered first, and suppose that there are k ways to cover it. Suppose further that the jth option for p ends with a secondary item sj, where {s_1, ..., sk} are distinct. Modify the algorithm so that, whenever a solution contains the jth option for p, it leaves items {s_1, ..., sj−1} uncovered. (In other words, the modified algorithm will emulate the behavior of the unmodified algorithm on a much larger instance, in which the jth option for p contains all of s_1, s_2, ..., sj.)

▸ 84. [25] Number the options of an XCC problem from 1 to M. A minimax solution is one whose maximum option number is as small as possible. Explain how to modify Algorithm C so that it determines all of the minimax solutions (omitting any that are known to be worse than a solution already found).

85. [22] Sharpen the algorithm of exercise 84 so that it produces exactly one minimax solution — unless, of course, there are no solutions at all.

▸ 86. [M25] Modify Algorithm C so that, instead of finding all solutions to a given XCC problem, it gives a Monte Carlo estimate of the number of solutions and the time needed to find them, using Theorem 7.2.2E. (Thus the modified algorithm is to Algorithm C as Algorithm 7.2.2E is to Algorithm 7.2.2B.)

87. [20] A double word square is an n × n array whose rows and columns contain 2n different words. Encode this problem as an XCC problem. Can you save a factor of 2 by not generating the transpose of previous solutions? Does Algorithm C compete with the algorithm of exercise 7.2.2–28 (which was designed explicitly to handle such problems)?

88. [21] Instead of finding all of the double word squares, we usually are more interested in finding the best one, in the sense of using only words that are quite common. For example, it turns out that a double word square can be made from the words of WORDS(1720) but not from those of WORDS(1719). Show that it’s rather easy to find the smallest W such that WORDS(W) supports a double word square, via dancing links.

89. [24] What are the best double word squares of sizes 2 × 2, 3 × 3, ..., 7 × 7, in the sense of exercise 88, with respect to The Official SCRABBLE® Players Dictionary? [Exercise 7.2.2–32 considered the analogous problem for symmetric word squares.]

▸ 90. [22] A word stair of period p is a cyclic arrangement of words, offset stepwise, that contains 2p distinct words across and down. They exist in two varieties, left and right:

What are the best five-letter word stairs, in the sense of exercise 88, for 1 ≤ p ≤ 10? Hint: You can save a factor of 2p by assuming that the first word is the most common.

91. [40] For given W, find the largest p such that WORDS(W) supports a word stair of period p. (There are two questions for each W, examining stairs to the {left, right}.)

92. [24] Some p-word cycles define two-way word stairs that have 3p distinct words:

What are the best five-letter examples of this variety, for 1 ≤ p ≤ 10?

93. [22] Another periodic arrangement of 3p words, perhaps even nicer than that of exercise 92 and illustrated here for p = 3, lets us read them diagonally up or down, as well as across. What are the best five-letter examples of this variety, for 1 ≤ p ≤ 10? (Notice that there is 2p-way symmetry.)

94. [20] (É. Lucas.) Find a binary cycle (x_0x_1 ... x_15) for which the 16 quadruples xkx(k+1) mod 16 x(k+3) mod 16 x(k+4) mod 16 for 0 ≤ k < 16 are distinct.

▸ 95. [20] Given 0 ≤ p < q ≤ n, explain how to use color controls and Algorithm C to find all cycles (x_0x_1 ... xm−1) of 0s and 1s, where m=Σk=pq(nk), with the property that the m binary vectors {x_0x_1 ... xn−1, x_1x_2 ... xn, ..., xm−1x_0 ... xn−2} are distinct and have weight between p and q. (In other words, all n-bit binary vectors y = y_1 ... yn with p ≤ νy ≤ q occur exactly once in the cycle. We studied the special case of de Bruijn cycles, for which p = 0 and q = n, in Section 7.2.1.1.)

For example, when n = 7, p = 0, and q = 3, the cycle

(0000000100000110000101000101100010010101001001100100011010000111)

exhibits all binary 7-tuples with a majority of 0s. When n = 7, p = 3, q = 4, the cycle

(0000111000101100011010010101011010100110011011001011100100111010001111)

shows all 7-tuples obtainable by removing the first bit of an 8-tuple with four 0s, four 1s.

Exactly how many cycles exist, when (n, p, q) = (7, 0, 3) or (7, 3, 4)? How long does it take for Algorithm C to find them?

96. [M20] Find an 8 × 8 binary torus whose sixty-four 2 × 3 subrectangles are distinct.

97. [M21] Find all 9 × 9 ternary ourotoruses D = (di,j) that are symmetrical, in the sense that d(i+3) mod 9, (j+3) mod 9 = (di,j + 1) mod 3. (See exercise 7.2.1.1–109.)

98. [25] Prove that the exact cover problem with color controls is NP-complete, even if every option consists of only two items.

99. [20] True or false: Every XCC problem can be reformulated as an ordinary exact cover problem with the same solutions and the same number of options.

▸ 100. [20] The general constraint satisfaction problem (CSP) is the task of finding all n-tuples x_1 ... xn that satisfy a given system of constraints C_1, ..., Cm, where each constraint is defined by a relation on a nonempty subset of the variables {x_1, ..., xn}.

For example, a unary constraint is a relation of the form xk ∈ Dk; a binary constraint is a relation of the form (xj, xk) ∈ Djk; a ternary constraint is a relation of the form (xi, xj, xk) ∈ Dijk; and so on.

1. Find all x_1x_2x_3x_4x_5 for which 0 ≤ x_1 ≤ x_2 ≤ x_3 ≤ x_4 ≤ x_5 ≤ 2 and x_1 +x_3 +x_5 = 3.

2. Formulate the problem of part (a) as an XCC problem.

3. Explain how to formulate any CSP as an XCC problem.

▸ 101. [25] (The zebra puzzle.) Formulate the following query as an XCC problem: “Five people, from five different countries, have five different occupations, own five different pets, drink five different beverages, and live in a row of five differently colored houses.

- The Englishman lives in a red house.

- The yellow house hosts a diplomat.

- The Norwegian’s house is the leftmost.

- The milk drinker lives in the middle house.

- The painter comes from Japan.

- The coffee-lover’s house is green.

- The dog’s owner is from Spain.

- The violinist drinks orange juice.

- The white house is just left of the green one.

- The Norwegian lives next to the blue house.

- The horse lives next to the diplomat.

- The Ukrainian drinks tea.

- The sculptor breeds snails.

- The nurse lives next to the fox.

Who trains the zebra, and who prefers to drink just plain water?”

▸ 102. [25] Explain how to find all solutions to a Japanese arrow puzzle with Algorithm C. (See exercise 7.2.2–68.)

▸ 103. [M28] Musical pitches in the Western system of “equal temperament” are the notes whose frequency is 440 · 2n/12 cycles per second, for some integer n. The pitch class of such a note is n mod 12, and seven of the twelve possible pitch classes are conventionally designated by letters:

0=A,2=B,3=C,5=D,7=E,8=F,10=G.

The other classes are named by appending sharp (♯) or flat (♭) signs, to go up or down by 1; thus 1 = A♯ = B♭, 4 = C♯ = D♭, ..., 11 = G♯ = A♭.

Arnold Schoenberg popularized a composition technique that he called a twelve-tone row, which is simply a permutation of the twelve pitch classes. For example, his student Alban Berg featured the motif

which is the twelve-tone row 8 7 3 0 10 5 11 4 6 9 1 2, in the first movement of his Lyric Suite (1926), and in another composition he had written in 1925.

In general we can say that an n-tone row x = x_0x_1 ... xn−1 is a permutation of {0, 1, ..., n−1}. Two n-tone rows x and x′ are considered to be equivalent if they differ only by a transposition — that is, if xk′=(xk+d) mod n for some d and for 0 ≤ k < n. Thus, the number of inequivalent n-tone rows is exactly (n − 1)!.

1. Berg’s 12-tone row above has the additional property that the intervals between adjacent notes, (xk − xk−1)mod n, are {1, ..., n − 1}. Prove that an n-tone row can have this all-interval property only if n is even and xn−1 = (x_0 + n/2) mod n.

2. Use Algorithm C to find n-tone rows with the all-interval property. How many inequivalent solutions arise, when 2 ≤ n ≤ 12?

3. Any all-interval n-tone row leads easily to several others. For example, if x = x_0x_1 ... xn−1 is a solution, so is its reversal xR = xn−1 ... x_1x_0; and so is cx = (cx_0 mod n)(cx_1 mod n) ... (cxn−1 mod n) whenever c ⊥ n. Prove that the cyclic shift xQ = xk ... xn−1x_0 ... xk−1 is also a solution, when xk − xk−1 = ±n/2.

4. True or false: In part (c) we always have xRQ = xQR.

5. The 12-tone row of Alban Berg shown above is symmetrical, because it is equivalent to xR. Other kinds of symmetry are also possible; for example, the row x = 0 1 3 7 2 5 11 10 8 4 9 6 is equivalent to −xQ. How many symmetrical all-interval n-tone rows exist, for n ≤ 12?

104. [M28] Assume that n +1 = p is prime. Given an n-tone row x = x_0x_1 ... xn−1, define yk = x(k−1) mod p whenever k is not a multiple of p, and let x(r) = yry2r ... ynr be the n-tone row consisting of “every rth element of x” (if xn is blank). For example, when n = 12, every 5th element of x is the sequence x(5) = x_4x_9x_1x_6x_11x_3x_8x_0x_5x_10x_2x_7.

An n-tone row is called perfect if it is equivalent to x(r) for 1 ≤ r ≤ n. For example, the amazing 12-tone row 0 1 4 2 9 5 11 3 8 10 7 6 is perfect.

1. Prove that a perfect n-tone row has the all-interval property.

2. Prove that a perfect n-tone row also satisfies x ≡ xR.

105. [22] Using the “word search puzzle” conventions of Figs. 71 and 72, show that the words ONE, TWO, THREE, FOUR, FIVE, SIX, SEVEN, EIGHT, NINE, TEN, ELEVEN, and TWELVE can all be packed into a 6 × 6 square, leaving one cell untouched.

106. [22] Also pack two copies of ONE, TWO, THREE, FOUR, FIVE into a 5 × 5 square.

▸ 107. [25] Pack as many of the following words as possible into a 9 × 9 array, simultaneously satisfying the rules of both word search and sudoku:

ACRECOMPARECORPORATEMACROMOTETROAMARTCOMPUTERCROPMETAPARAMETERTAME

▸ 108. [32] The first 44 presidents of the U.S.A. had 38 distinct surnames: ADAMS, ARTHUR, BUCHANAN, BUSH, CARTER, CLEVELAND, CLINTON, COOLIDGE, EISENHOWER, FILLMORE, FORD, GARFIELD, GRANT, HARDING, HARRISON, HAYES, HOOVER, JACKSON, JEFFERSON, JOHNSON, KENNEDY, LINCOLN, MADISON, MCKINLEY, MONROE, NIXON, OBAMA, PIERCE, POLK, REAGAN, ROOSEVELT, TAFT, TAYLOR, TRUMAN, TYLER, VANBUREN, WASHINGTON, WILSON.

1. What’s the smallest square into which all of these names can be packed, using word search conventions, and requiring all words to be connected via overlaps?

2. What’s the smallest rectangle, under the same conditions?

▸ 109. [28] A “wordcross puzzle” is the challenge of packing a given set of words into a rectangle under the following conditions: (i) All words must read either across or down, as in a crossword puzzle. (ii) No letters are adjacent unless they belong to one of the given words. (iii) The words are rookwise connected. (iv) Words overlap only when one is vertical and the other is horizontal. For example, the eleven words ZERO, ONE, ..., TEN can be placed into an 8 × 8 square under constraints (i) and (ii) as shown; but (iii) is violated, because there are three different components.

Explain how to encode a wordcross puzzle as an XCC problem. Use your encoding to find a correct solution to the problem above. Do those eleven words fit into a smaller rectangle, under conditions (i), (ii), and (iii)?

110. [30] What’s the smallest wordcross square that contains the surnames of the first 44 U.S. presidents? (Use the names in exercise 108, but change VANBUREN to VAN BUREN.)

111. [21] Find all 8 × 8 crossword puzzle diagrams that contain exactly (a) 12 3-letter words, 12 4-letter words, and 4 5-letter words; (b) 12 5-letter words, 8 2-letter words, and 4 8-letter words. They should have no words of other lengths.

▸ 112. [28] A popular word puzzle in Brazil, called ‘Torto’ (‘bent’), asks solvers to find as many words as possible that can be traced by a noncrossing king path in a given 6 × 3 array of letters. For example, each of the words THE, MATURE, ART, OF, COMPUTER, and PROGRAMMING can be found in the array shown here.

1. Does that array contain other common words of eight or more letters?

2. Create a 6 × 3 array that contains TORTO, WORDS, SOLVER, and many other interesting English words of five or more letters. (Let your imagination fly.)

3. Is it possible to pack ONE, TWO, THREE, ..., EIGHT, NINE, TEN into a Torto array?

▸ 113. [21] An ‘alphabet block’ is a cube whose six faces are marked with letters. Is there a set of five alphabet blocks that are able to spell the 25 words TREES, NODES, STACK, AVAIL, FIRST, RIGHT, ORDER, LISTS, GIVEN, LINKS, QUEUE, GRAPH, TIMES, BLOCK, VALUE, TABLE, FIELD, EMPTY, ABOVE, POINT, THREE, UNTIL, HENCE, QUITE, DEQUE? (Each of these words appears more than 50 times in Chapter 2.)

114. [M25] Let α be a permutation of the cells of a 9 × 9 array that takes any sudoku solution into another sudoku solution. We say that α is an automorphism of the sudoku solution S = (sij) if there’s a permutation π of {1, 2, ..., 9} such that s(ij)α = sijπ for 0 ≤ i, j < 9. For example, the permutation that takes ij into (ij)α = ji, commonly called transposition, is an automorphism of (28b), with respect to the permutation π = (24)(37)(68); but it is not an automorphism of (28a) or (28c).

Show that Algorithm C can be used to find all sudoku solutions that have a given automorphism α, by defining an appropriate XCC problem.

How many sudoku solutions have transposition as an automorphism?

115. [M25] Continuing exercise 114, how many hypersudoku solutions have auto-morphisms of the following types? (a) transposition; (b) the transformation of exercise 67(d); (c) 90° rotation; (d) both (b) and (c).

▸ 116. [M25] Given a graph G on vertices V, let μ(G) be obtained by (i) adding new vertices V′ = {v′ | v ∈ V}, with u′ — v when u — v; and also (ii) adding another vertex w, with w — v′ for all v′ ∈ V′. (If G has m edges and n vertices, μ(G) has 3m + n edges and 2n + 1 vertices.) The Mycielski graphs Mc are defined for all c ≥ 2 by setting M_2 = K_2 and Mc+1 = μ(Mc); they have 7183c−342c+12 edges and 342c−1 vertices:

1. Prove that each Mc is triangle-free (contains no subgraph K_3).

2. Prove that the chromatic number χ(Mc) = c.

3. Prove that each Mc is in fact “χ-critical”: Removing any edge decreases χ.

▸ 117. [24] (Graph coloring.) Suppose we want to find all possible ways to label the vertices of graph G with d colors; adjacent vertices should have different colors.

1. Formulate this as an exact cover problem, with one primary item for each vertex and with d secondary items for each edge.

2. Sometimes G’s edges are conveniently specified by giving a family {C_1, ..., Cr} of cliques, where each Cj is a subset of vertices; then u — v if and only if u ∈ Cj and v ∈ Cj for some j. (For example, the 728 edges of the queen graph Q_8 can be specified by just 8+8+13+13 = 42 cliques — one clique for each row, column, and diagonal.) Modify the construction of (a) so that there are only rd secondary items.

3. In how many ways can Q_8 be 9-colored? (Compare method (a) to method (b).)

4. Each solution to the coloring problem that uses k different colors is obtained dk¯=d(d−1)...(d−k+1) times, because of the symmetry between colors. Modify (a) and (b) so that each essentially different solution is obtained just once, when the symmetry-breaking technique of exercise 122 is used.

5. In how many ways can the Mycielski graph Mc be c-colored, for 2 ≤ c ≤ 5?

6. Use Algorithm C to verify that Mc can’t be (c − 1)-colored, for 2 ≤ c ≤ 5.

7. Try (c − 1)-coloring Mc when a random edge is removed, for 2 ≤ c ≤ 5.

118. [21] (Hypergraph coloring.) Color the 64 cells of a chessboard with four colors, so that no three cells of the same color lie in a straight line of any slope.

119. [21] Show that all solutions to the problem of placing MacMahon’s 24 triangles (58) into a hexagon with all-white border can be rotated and reflected so that the all-white triangle has the position that it occupies in (59b). Hint: Factorize.

120. [M29] Section 2.3.4.3 discussed Hao Wang’s “tetrad tiles,” which are squares that have specified colors on each side. Find all ways in which the entire plane can be filled with tiles from the following families of tetrad types, always matching colors at the edges where adjacent tiles meet [see Scientific American 231, 5 (Nov. 1965), 103, 106]:

(The tetrad tiles must not be rotated or flipped.) Hint: Algorithm C will help.

▸ 121. [M29] Exercise 2.3.4.3–5 discusses 92 types of tetrads that are able to tile the plane, and proves that no such tiling is toroidal (periodic).

1. Show that the tile called βUS in that exercise can’t be part of any infinite tiling. In fact, it can appear in only n + 1 cells of an m × n array, when m, n ≥ 4.

2. Show that, for all k ≥ 1, there’s a unique (2k − 1) × (2k − 1) tiling for which the middle tile is δRD. (Consequently, by the infinity lemma, there’s a unique tiling of the entire plane in which δRD is placed at the origin.)

3. Similarly, show that there are exactly (2, 3, 3, 57) tilings of size (2k − 1) × (2k − 1) whose middle tile is respectively (δRU, δLD, δLU, δSU), for all k ≥ 3.

4. How many tilings of the infinite plane have (δRU, δLD, δLU, δSU) at the origin?

▸ 122. [28] Extend Algorithm C so that it finds only 1/d! of the solutions, in cases where the input options are totally symmetric with respect to d of the color values, and where every solution contains each of those color values at least once. Assume that those values are {v, v + 1, ..., v + d − 1}, and that all other colors have values < v. Hint: Modify the algorithm so that the first such color it assigns is always v, then v + 1, etc.

123. [M20] Apply the algorithm of exercise 122 to the following toy problem with parameters m and n: There are n primary items pk and n secondary items qk, for 1 ≤ k ≤ n; and there are mn options, ‘pk qk:j’ for 1 ≤ j ≤ m and 1 ≤ k ≤ n. (The solutions to this problem are the mappings of {1, ..., n} into {1, ..., m}, which may also be regarded as the partitions of {1, ..., n} into parts labeled {1, ..., m}.) Algorithm C will obviously find mn={n1}m1¯+{n1}m2¯+...+{nm}mm¯ solutions. But the modified algorithm finds only the “unlabeled” partitions, of which there are {n1}+{n2}+...+{nm}.

▸ 124. [M22] Devise a system of coordinates for representing the positions of equilateral triangles in patterns such as (59). Represent also the edges between them.

125. [M20] When a set of s triangles is magnified by an integer k, we obtain sk^2 triangles. Describe the coordinates of those triangles, in term of the coordinates of the originals, using the system of exercise 124.

126. [23] Find all solutions of MacMahon’s problem (59), by applying Algorithm C to a suitable set of items and options based on the coordinate system in exercise 124. How much time is saved by using the improved algorithm of exercise 122?

127. [M28] There are 4^12 ways to prescribe the border colors of a hexagon like those in (59). Which of them can be completed to a color-matched placement of all 24 triangles?

▸ 128. [25] Eleven of MacMahon’s triangles (58) involve only the first three colors (not black). Arrange them into a pleasant pattern that tiles the entire plane when replicated.

▸ 129. [M34] The most beautiful patterns that can be made with MacMahon’s triangles are those with attractive symmetries, which can be of two kinds: strong symmetry (a rotation or reflection that doesn’t change the pattern, except for permutation of colors) or weak symmetry (a rotation or reflection that preserves the “color patches,” the set of boundaries between different colors).

Exactly how many essentially different symmetrical patterns are possible, in a hexagon?

130. [21] Partition MacMahon’s triangles (58) into three sets of eight, each of which can be placed on the faces of an octahedron, with matching edge colors.

131. [28] (P. A. MacMahon, 1921.) Instead of using the colored tiles of (58), which yield (59), we can form hexagons from 24 different triangles in two other ways:

The left diagram shows a “jigsaw puzzle” whose pieces have four kinds of edges. The right diagram shows “triple three triominoes,” which have zero, one, two, or three spots at each edge; adjacent triominoes should have a total of three spots where they meet.

1. In how many ways can that jigsaw puzzle make a hexagon? (All pieces are white.)

2. How many triomino arrangements have that pattern of dots at the edges?

132. [40] (W. E. Philpott, 1971.) There are 4624 = 68^2 tiles in a set that’s like (58), but it uses 24 different colors instead of 4. Can they be assembled into an equilateral triangle of size 68, with constant color on the boundary and with matching edges inside?

133. [21] (P. A. MacMahon, 1921.) A set of 24 square tiles can be constructed, analogous to the triangular tiles of (58), if we restrict ourselves to just three colors. For example, they can be arranged in a 4 × 6 rectangle as shown, with all-white border. In how many ways can this be done?

134. [23] The nonwhite areas of the pattern in exercise 133 form polyominoes (rotated 45°); in fact, the lighter color has an S pentomino, while the darker color has both P and V. How often do each of the twelve pentominoes occur, among all of the solutions?

135. [23] (H. L. Nelson, 1970.) Show that MacMahon’s squares of exercise 133 can be used to wrap around the faces of a 2 × 2 × 2 cube, matching colors wherever adjacent.

▸ 136. [HM28] (J. H. Conway, 1958.) There are twelve ways to label the edges of a pentagon with {0, 1, 2, 3, 4}, if we don’t consider rotations and reflections to be different:

Cover a dodecahedron with these tiles, matching edge numbers. (Reflections are OK.)

137. [22] A popular puzzle called Drive Ya Nuts consists of seven “hex nuts” that have been decorated with permutations of the numbers {1, 2, 3, 4, 5, 6}. The object is to arrange them as shown, with numbers matching at the edges.

1. Show that this puzzle has a unique solution, with that particular set of seven. (Reflections of the nuts are not OK!)

2. Can those seven nuts form the same shape, but with the label numbers summing to 7 where they meet ({1, 6}, {2, 5}, or {3, 4})?

3. Hex nuts can be decorated with {1, 2, 3, 4, 5, 6} in 5! = 120 different ways. If seven of them are chosen at random, what’s the approximate probability that they define a puzzle with a unique solution, under matching condition (a)?

4. Find seven hex nuts that have a unique solution under both conditions (a) and (b).

138. [25] (Heads and tails.) Here’s a set of 24 square tiles that MacMahon missed(!):

They each show two “heads” and two “tails” of triangles, in four colors that exhibit all possible permutations, with heads pointing to tails. The tiles can be rotated, but not flipped over. We can match them properly in many ways, such as

where the 4 × 6 arrangement will tile the plane; the 5 × 5 arrangement has a special “joker” tile in the middle, containing all four heads.

1. How many 4 × 6 arrangements will tile the plane? (Consider symmetries.)

2. Notice that the half-objects at the top, bottom, left, and right of the 5 × 5 arrangement match the heads in the middle. How many such arrangements are possible?

3. Devise a 5 × 5 arrangement that will tile the plane, in conjunction with the 5 × 5 pattern shown above. Hint: Use an “anti-joker” tile, which contains all four tails.

139. [M25] Excellent human-scale puzzles have been made by choosing nine of the 24 tiles in exercise 138, redrawing them with whimsical illustrations in place of the triangles, and asking for a 3 × 3 arrangement in which heads properly match tails.

1. How many of the (249) choices of 9 tiles lead to essentially different puzzles?

2. How many of those puzzles have exactly k solutions, for k = 0, 1, 2, ... ?

140. [29] (C. D. Langford, 1959.) MacMahon colored the edges of his tiles, but we can color the vertices instead. For example, we can make two parallelograms, or a truncated triangle, by assembling the 24 vertex-colored analogs of (58):

Such arrangements are much rarer than those based on edge matching, because edges are common to only two tiles but vertices might involve up to six.

1. In how many essentially distinct ways can those shapes be formed?

2. The first parallelogram is a scaled-up version of the “straight hexiamond” , with dimensions doubled. How many of the other eleven scaled-up hexiamond shapes can be assembled from Langford’s tiles? (See exercises 125 and 309.)

3. Each of the seven tetrahexes also yields an interesting shape that consists of 24 triangles. (See exercise 316.) How do Langford’s tiles behave in those shapes?

141. [24] Combining exercises 133 and 140, we can also adapt MacMahon’s 24 tri-colored squares to vertex matching instead of edge matching. Noteworthy solutions are

1. In how many essentially different ways can those 24 tiles be properly packed into rectangles of these sizes, leaving a hole in the middle of the 5 × 5?

2. Discuss tiling the plane with such solutions.

▸ 142. [23] (Zdravko Zivkovic, 2008.) Edge and vertex matching can be combined into a single design if we replace MacMahon’s 24 squares by 24 octagons. For example,

illustrate 4 × 6 arrangements in which there’s vertex matching in the (i) left half, (ii) bottom half, or (iii) northwest and southeast quadrants, while edge matching occurs elsewhere. (We get vertex matching when an octagon’s center is , edge matching when it’s .) How many 4 × 6 arrangements satisfy (i), (ii), and (iii), respectively?

▸ 143. [M25] The graph simplex (n, a, b, c, 0, 0, 0) in the Stanford GraphBase is the truncated triangular grid consisting of all vertices xyz such that x + y + z = n, 0 ≤ x ≤ a, 0 ≤ y ≤ b, and 0 ≤ z ≤ c. Two vertices are adjacent if their coordinates all differ by at most 1. The boundary edges always define a convex polygon. For example, simplex (7, 7, 5, 3, 0, 0, 0) is illustrated here.

1. What simplex graphs correspond to the three shapes in exercise 140?

2. The examples in (a) have 24 interior triangles, but simplex (7, 7, 5, 3, 0, 0, 0) has 29. Can any other convex polygons be made from 24 triangles, connected edgewise?

3. Design an efficient algorithm that lists all possible convex polygons that can be formed from exactly N triangles, given N. Hint: Every convex polygon in a triangular grid can be characterized by the six numbers in its boundary path x_0x_1x_2x_3x_4x_5, which moves xk steps in direction (60k)° for k = 0, 1, ..., 5. For example, the boundary of simplex (7, 7, 5, 3, 0, 0, 0) is 503412.

4. Can every convex polygon in a triangular grid be described by a simplex graph?

144. [24] The idea of exercise 142 applies also to triangles and hexagons, allowing us to do both vertex and edge matching with yet another set of 24 tiles:

Here there’s vertex matching in the bottom five tiles of (i), and in the upper left five and bottom five of (ii), with edge matching elsewhere. In how many ways can the big hexagon be made from these 24 little hexagons, under constraints (i) and (ii)?

▸ 145. [M20] Many problems that involve an l × m × n cuboid require a good internal representation of its (l+1)(m+1)(n+1) vertices, its l(m+1)(n+1) + (l+1)m(n+1) + (l+1)(m+1)n edges, and its lm(n+1)+l(m+1)n+(l+1)mn faces, in addition to its lmn individual cells. Show that there’s a convenient way to do this with integer coordinates (x, y, z) whose ranges are 0 ≤ x ≤ 2l, 0 ≤ y ≤ 2m, 0 ≤ z ≤ 2n.

▸ 146. [M30] There are 30 ways to paint the colors {a, b, c, d, e, f} on the faces of a cube:

(If a is on top, there are five choices for the bottom color, then six cyclic permutations of the remaining four.) Here’s one way to arrange six differently painted cubes in a row, with distinct colors on top, bottom, front, and back (as in “Instant Insanity”), and with the further proviso that adjacent cubes have matching colors where they share a face:

1. Explain why any such arrangement also has the same color at the left and right.

2. Invent a way to name each cube, distinguishing it from the other 29.

3. How many essentially different arrangements like (*) are possible?

4. Can all 30 cubes be used to make five such arrangements simultaneously?

147. [30] The 30 cubes of exercise 146 can be used to make “bricks” of various sizes l × m × n, by assembling l · m · n of them into a cuboid that has solid colors on each exterior face, as well as matching colors on each interior face. For example, each cube naturally joins with its mirror image to form a 1 × 1 × 2 brick. Two such bricks can then join up to make a 1 × 2 × 2; the one illustrated here has a in front, b in the back, c at the left and right, d at the top, and e at the bottom.

1. Assemble all 30 cubes into a magnificent brick of size 2 × 3 × 5.

2. Compile a catalog of all the essentially different bricks that can be made.

148. [24] Find all the distinct cubes whose faces are colored a, b, or c, when opposite faces are required to have different colors. Then arrange them into a symmetric shape (with matching colors wherever they are in contact).

149. [M22] (Vertex-colored tetrahedra.) The graph simplex (3, 3, 3, 3, 3, 0, 0) is a tetrahedron of side 3 with 20 vertices. It has 60 edges, which come from 10 unit tetrahedra.

There are ten ways to color the vertices of a unit tetrahedron with four of the five colors {a, b, c, d, e}, because mirror reflections are distinct. Can those ten colored tetrahedra be packed into simplex (3, 3, 3, 3, 3, 0, 0), with matching colors at every vertex?

150. [23] Here’s a classic 19th century puzzle that was the first of its kind: “Arrange all the pieces to fill the square ... so that all the links of the Chain join together, forming an Endless Chain. The Chain may be any shape, so long as all the links join together, and all the pieces are used. This Puzzle can be done several different ways.”

(The desired square is 8 × 8.) In exactly how many different ways can it be solved?

▸ 151. [30] (Path dominoes.) A domino has six natural attachment points on its boundary, where we could draw part of a path that connects to neighboring dominoes. Thus (62)=15 different partial paths could potentially be drawn on it. However, only 9 distinct domino patterns with one subpath actually arise, because the 15 possibilities are reduced under 180° rotation to six pairs, plus three patterns that have central symmetry. Similarly, there are 27 distinct domino patterns that contain two partial paths (where the paths might cross each other). An 8 × 9 arrangement, which nicely illustrates all 36 of the possibilities, is shown; notice that its path is a Hamiltonian cycle, consisting of a single loop.

1. Only two of the dominoes in the arrangement above are in horizontal position. Find a single-loop 8 × 9 arrangement that has 18 horizontals and 18 verticals.

2. Similarly, find an arrangement that has the maximum number of horizontals.

152. [30] The complete set of path dominoes includes also twelve more patterns:

Arrange all 48 of them in an 8 × 12 array, forming a single loop.

153. [25] Here are six of the path dominoes, plus a “start” piece and a “stop” piece:

1. Place them within a 4 × 5 array so that they define a path from “start” to “stop.”

2. How many distinct “start” or “stop” pieces are possible, if they’re each supposed to contain a single subpath together with a single terminal point?

3. Design an eight-piece puzzle that’s like (a), but it involves four of the two-subpath dominoes instead of only two. (Your puzzle should have a unique solution.)

154. [M30] (C. R. J. Singleton, 1996.) After twelve days of Christmas, the person who sings a popular carol has received twelve partridges in pear trees, plus eleven pairs of humming birds, ..., plus one set of twelve drummers drumming, from his or her true love. Therefore an “authentic” partridge puzzle should try to pack (n+1−k) squares of size k × k, for 1 ≤ k ≤ n, into a box that contains P (n) = n · 1^2 +(n − 1) · 2^2 + ··· +1 · n^2 cells. For which values of n is P (n) a perfect square?

155. [20] That “authentic” partridge puzzle has a square solution when n = 6.

1. Exactly how many different solutions does it have in that case?

2. The affinity score of a partridge packing is the number of internal edges that lie on the boundary between two squares of the same size. (In (62) the scores are 165 and 67.) What solutions to (a) have the maximum and minimum affinity scores?

▸ 156. [30] Straightforward backtracking will solve the partridge puzzle for n = 8, using bitwise techniques to represent a partially filled 36 × 36 square in just 36 octabytes, instead of by treating it as the huge MCC problem (61) and applying a highly general solver such as Algorithm M. Compare these two approaches, by implementing them both. How many essentially different solutions does that partridge puzzle have?

157. [22] Complete the study of small partridges by extending (6_3) to n = 6 and 7.

158. [23] Another variation of the partridge puzzle when 2 ≤ n ≤ 7 asks for the smallest rectangular area that will contain k nonoverlapping squares of size k × k for 1 ≤ k ≤ n. For example, here are solutions for n = 2, 3, and 4:

(To show optimality for n = 4 one must prove that rectangles of sizes 6 × 17, 8 × 13, 5 × 21, and 7 × 15 are too small.) Solve this puzzle for n = 5, 6, and 7.

▸ 159. [21] Suggest a way to speed up the text’s solution to the 5-queens problem, by using the symmetries of a square to modify the items and options of (6_4).

160. [21] The 5-queens problem leads to an interesting graph, whose vertices are the 4860 solutions, with u — v when we can get from u to v by moving one queen. How many connected components does this graph have? Is one of them a “giant”?

▸ 161. [23] Three restricted queen-domination problems are prominent in the literature:

1. No two queens of a solution attack each other.

2. Each queen of a solution is attacked by at least one of the others.

3. The queens of a solution form a clique.

(The third and fourth examples in (6_5) are instances of types (ii) and (i).)

Explain how to formulate each of these variants as an MCC problem, analogous to (64). How many solutions of each type are present in the 5-queens problem?

162. [24] Say that a 𝒬n is an n × n array of n nonattacking queens. Sometimes a 𝒬n contains a 𝒬m for m < n; for example, eight of the possible 𝒬_5’s contain a 𝒬_4, and the 𝒬_17 illustrated here contains both a 𝒬_4 and a 𝒬_5.

What is the smallest n such that at least one 𝒬n contains (a) two 𝒬_4’s? (b) three 𝒬_4’s? (c) four 𝒬_4’s? (d) five 𝒬_4’s? (e) two Q5’s? (f) three 𝒬_5’s? (g) four 𝒬_5’s? (h) two 𝒬_6’s? (i) three 𝒬_6’s?

163. [20] Explain the peculiar rule for setting p in (71).

164. [17] When Algorithm M finds a solution x_0x_1 ... xl−1 in step M2, some of the nodes xj might represent the fact that some primary item will appear in no further options. Explain how to handle this “null” case, by modifying answer 13.

165. [M30] Consider an MCC problem in which we must choose 2 of 4 options to cover item 1, and 5 of 7 options to cover item 2; the options don’t interact.

1. What’s the size of the search tree if we branch first on item 1, then on item 2? Would it better to branch first on item 2, then on item 1?

2. Generalize part (a) to the case when item 1 needs p of p + d options, while item 2 needs q of q + d options, where q > p and d > 0.

166. [21] Extend answer 9 to the more general situation that arises in Algorithm M:

1. Let θp be the number of different choices that will be explored at the current position of the search tree if primary item p is selected for branching. Express θp as a function of LEN(p), SLACK(p), and BOUND(p).

2. Suppose θp = θp′ and SLACK(p) = SLACK(p′) = 0, but LEN(p) < LEN(p′). Should we prefer to branch on p or on p′, based on exercise 165?

167. [24] Let Mp be the number of options that involve the primary item p in a given MCC problem, and suppose that the upper bound vp for p’s multiplicity is ≥ Mp. Does the precise value of this upper bound affect the behavior of Algorithm M? (In other words, does vp = ∞ lead to the same running time as vp = Mp?)

▸ 168. [15] An MCC problem might have two identical options α, whose items are allowed to occur more than once. In such cases we might want the second copy of α to be in the solution only if the first copy is also present. How can that be achieved?

▸ 169. [22] Let G be a graph with n vertices. Formulate the problem of finding all of its t-element independent sets as an MCC problem with 1 + n items and n options.

170. [22] Continuing exercise 169, generate all of G’s t-element kernels — its maximal independent sets. (Your formulation will now need additional items and options.)

171. [25] Label the vertices of the Petersen graph with ten 5-letter words, in such a way that vertices are adjacent if and only if their labels have a common letter.

▸ 172. [29] A snake-in-the-box path in a graph G is a set U of vertices for which the induced graph G | U is a path. (Thus there are start/stop vertices s ∈ U and t ∈ U that each have exactly one neighbor in U; every other vertex of U has exactly two neighbors in U; and G | U is connected.)

For example, let G = P_4 ⊠ P_4 be the graph of king moves on a 4 × 4 board. The set of kings illustrated at the right is not a snake-in-the-box path in G; but it becomes one if we remove the king in the corner.

1. Use Algorithm M to discover all of the longest snake-in-the-box paths that are possible on an 8 × 8 chessboard, when G is the graph of all (i) king moves; (ii) knight moves; (iii) bishop moves; (iv) rook moves; (v) queen moves.

2. Similarly, a snake-in-the-box cycle is a set for which G | U is a cycle. (In other words, that induced graph is connected and 2-regular.) What are the longest possible snake-in-the-box cycles for those five chess pieces?

▸ 173. [30] (Knight and bishop sudoku.) Diagram (i) shows 27 knights, arranged with three in each row, three in each column, and three in each 3 × 3 box. Each of them has been labeled with the number of others that are a knight’s move away. Diagram (ii) shows 8 of them, from which the positions of the other 19 can be deduced. Diagrams (iii) and (iv) are analogous, but for bishops instead of knights: (iii) solves puzzle (iv). (i)

1. Explain how to find all completions of such diagrams using Algorithm M.

2. Find the unique completions of the following puzzles:

3. Compose additional puzzles like those of (b), in which all clues have the same numerical labels. Try to use as few clues as possible.

4. Construct a uniquely solvable knight sudoku puzzle that has only three clues.

174. [35] (Nikolai Beluhov, 2019.) Find a uniquely solvable sudoku puzzle with nine labeled knights that remains uniquely solvable when the knights are changed to bishops.

▸ 175. [M21] Given an M × N matrix A = (aij) of 0s and 1s, explain how to find all vectors x = (x_1 ... xM) with 0 ≤ xi ≤ ai for 1 ≤ i ≤ M such that xA = (y_1 ... yN), where uj ≤ yj ≤ vj for 1 ≤ j ≤ N. (This generalizes the MCC problem by allowing the ith option to be repeated up to ai times.)

▸ 176. [M25] Given an M × N matrix A = (aij) of 0s, 1s, and 2s, explain how to find all subsets of its rows that sum to exactly (a) 2 (b) 3 (c) 4 (d) 11 in each column, by formulating those tasks as MCC problems.

177. [M21] Algorithm 7.2.1.5M generates the p(n_1, ..., nm) partitions of the multiset {n_1 · x_1, ..., nm · xm} into submultisets. Consider the special cases where n_1 = ··· = ns = 1 and ns+1 = ··· = ns+t = 2 and s + t = m.

1. Generate those partitions with Algorithm M, using the previous two exercises.

2. Also generate the q(n_1, ..., nm) multipartitions into distinct multisets.

178. [M22] (Factorizations of an integer.) Use Algorithm M to find all representations of 360 as a product n_1 · n_2 · ... · nt, where (a) 1 < n_1 < ··· < nt; (b) 2 ≤ n_1 ≤ ··· ≤ nt.

179. [15] By removing duplicate rows and columns, matrix A reduces to A′:

A=(100000011100110111011100000011000011101000010111111100);A′=(1000011011010001101001011110).

Derive the exact covers of A from the exact covers of A′.

▸ 180. [M28] (D. Eppstein, 2008.) Prove that every strict exact cover problem with parameters 1 ≤ t′ ≤ t, as defined in (74), contains t′ items i_1, ..., it′ and t+t′−1 options
op=‘i1...ip’, for 1≤p≤t′;op+q=‘...it′...’, for 1≤q<t.

Furthermore, ir ∈ op+q if and only if 1 ≤ q < t − r − t′, for 1 ≤ r ≤ t′.

181. [M20] Find constants cr such that D(5n+r)=4ncr−13 for n≥3 and 0 ≤ r < 5.

182. [21] (D. Eppstein, 2008.) Find a strict exact cover problem with 8 options, whose search tree contains 16 nodes and 7 solutions.

183. [46] Let D^(n) be the maximum number of nodes in Algorithm X’s search tree, taken over all strict exact cover problems with n options. What is lim supn→∞ D^(n)1/n?

▸ 184. [M22] Suppose 0 ≤ t ≤ ϖn. Is there a strict exact cover problem with n items that has exactly t solutions? (For example, consider the case n = 9, t = 10000.)

185. [M23] What is the largest number of solutions to a strict exact cover problem that has N_1 primary items and N_2 secondary items?

186. [M24] Consider l = 0 when Algorithm X is given the extreme problem of order n.

1. How many updates, un, does it perform when covering i in step X4?

2. How many does it perform in step X5, when the option containing x_0 has size k?

3. Therefore derive (84).

187. [HM29] Let X(z) = ∑n xnzn/n! generate the sequence ⟨xn⟩ of (82).

1. Use (84) to prove that X(z)=eez ∫0z((2t−1)e4t−(t−1)e3t+2te2t+et)e−et dt.

2. Let Tr,s(z)=eez ∫0ztr est e−et dt. Prove that Tr,0(z)/r! generates ⟨an,r+1⟩ in (83).

3. Show that Tr,0(z) = (Tr+1, 1(z)+ zr+1)/(r + 1); furthermore, when s > 0,

Tr,s(z)=(Σk=0r(−1)krk¯sk+1(Tr−k,s+1(z)+zr−kesz))−(−1)rr!sr+1eez−1.

4. Therefore X(z) = 22eez−1 + 12T0,0(z) − (2z − 1)e3z − 5ze2z − (12z + 5)ez − 12z − 18.

▸ 188. [M21] Prove that the Gould numbers 〈ϖn^〉=〈0,1,1,3,9,31,121,523,2469,...〉 can be calculated rapidly by forming a triangle of numbers analogous to Peirce’s triangle 7.2.1.5–(12):

Here the entries ϖ^n1,ϖ^n2,...,ϖ^nn of the nth row obey the simple recurrence
ϖ^nk=ϖ^(n−1)k+ϖ^n(k+1), if 1≤k<n;ϖ^nn=ϖ^(n−1)1, if n>2;
and initially ϖ^11=0,ϖ^22=1. Hint: Give a combinatorial interpretation of ϖ^nk.

189. [HM34] Let ρn=ϖ^n−g^ϖn (see (86)). We’ll prove that |ρn| = O(e−n/ln^2n ϖn), by applying the saddle point method to R(z)=Σn ρnzn/n!=eez ∫z∞e−et dt. The idea is to show that |R(z)| is rather small when z = ξeiθ where ξeξ = n as in 7.2.1.5–(24).

1. Express |eez| and |e−ez| in terms of x and y when z = x + iy.

2. If 0≤θ≤π2,y=ξ sin θ≤32,0<c1<cos 32, prove |R(ξeiθ)|=O(exp(eξ−c1eξ)).

3. If 0≤θ≤π2,y=ξ sin θ≤32,0<c2<98, prove |R(ξeiθ)|=O(exp(eξ−c2eξ/ξ)).

4. Consequently ρn−1/ϖn−1 = O(e−n/ln^2n), as desired.

190. [HM46] Study the signs of the residual quantities ρn=ϖn−g^ϖn in exercise 189.

191. [HM22] The length of the tail of a random set permutation is known to have a probability distribution whose generating function is G(z)=∫0∞e−x(1+x)zdx−1=Σk=1∞ g^kzk. (The first few probabilities in this distribution are (g^1,g^2,...,g^9)≈(.59635,.26597,.09678,.03009,.00823,.00202,.00045,.00009,.00002); see answer 189.) What is the average length? What is the variance?

192. [HM29] What’s the asymptotic value of g^n when n is large?

193. [M21] Why do (87) and (88) count updates when matching in complete graphs?

194. [HM23] Consider recurrences of the form X(t+1) = at + tX(t−1). For example, at = 1 yields the total number of nodes in the search tree for matching Kt+1.

1. Prove that 1 + 2q +(2q)(2q − 2) + ··· +(2q)(2q − 2) ... (2) = ⌊e1/22qq!⌋.

2. Find a similar “closed formula” for 1 + (2q − 1) + (2q − 1)(2q − 3) + ··· + (2q − 1) · (2q − 3) ... (3)(1). Hint: Use the fact that ex erf(x)=Σn≥0xn+1/2/(n+1/2)!.

3. Estimate the solution U(2q + 1) of (87) to within O(1).

4. Similarly, give a good approximation to the solution U(2q) of (88).

▸ 195. [M22] Approximately how many updates does Algorithm X perform, when it is asked to find all of the perfect matchings of the graph (89)?

▸ 196. [M29] Given a bounded permutation problem defined by a_1 ... an, consider the “dual” problem defined by b_1 ... bn, where bk is the number of j such that 1 ≤ j ≤ n and aj ≥ n + 1 − k. [Equivalently, bn ...b_1 is the conjugate of the integer partition an ...a_1, in the sense of Section 7.2.1.4.]

1. What is the dual problem when n = 9 and a_1 ... a_9 = 246677889?

2. Prove that the solutions to the dual problem are essentially the inverses of the permutations that solve the original problem.

3. If Algorithm X begins with an a_1-way branch on item X_1, how many updates does it perform while preparing for the subproblems at depth 1 of its search tree?

4. How many solutions does a bounded permutation problem have, given a_1 ... an?

5. Give a formula for the total number of updates, assuming that the algorithm always branches on Xj at depth j − 1 of the search tree.

6. Evaluate the formula of (e) when aj = n for 1 ≤ j ≤ n (that is, all permutations).

7. Evaluate the formula of (e) when aj = min(j+1, n) for 1 ≤ j ≤ n.

8. Evaluate the formula of (e) when aj = min(2j, n) for 1 ≤ j ≤ n.

9. Show, however, that the assumption in (e) is not always correct. How can the total updates be calculated correctly in general?

197. [M25] Let P (a_1, ..., an) be the set of all permutations p_1 ... pn that solve the bounded permutation problem for a_1 ... an, given a_1 ≤ a_2 ≤ ··· ≤ an and aj ≥ j.

1. Prove that P (a_1, ..., an) = {(ntn) ... (2t_2)(1t_1) | j ≤ tj ≤ an for 1 ≤ j ≤ n}.

2. Also prove that P (a_1, ..., an) = {σntn ... σ2t_2 σ1t_1 | j ≤ tj ≤ an for 1 ≤ j ≤ n}, where σst is the (t +1 − s)-cycle (t t−1 ... s+1 s).

3. Let C(p) be the number of cycles in the permutation p, and let I(p) be the number of inversions. Find the generating functions

C(a1,...,an)=Σp∈P(a1,...an)zC(p)andI(a1,...,an)=Σp∈P(a1,...an)zI(p).

198. [M25] Let πrs = Pr(pr = s), when p is a random element of P (a_1, ..., an).

1. Compute these probabilities when n = 9 and a_1a_2 ... a_9 = 255667999.

2. If r < r′ and s < s′, show that πrs/πrs′ = πr′ s/πr′ s′, when πrs′ πr′ s′ ≠ 0.

199. [M25] Analyze the behavior of Algorithm X on the 3D matching problem whose options are ‘ai bj ck’ for 1 ≤ i, j ≤ n and 1 ≤ k ≤ (i ≤ m? m − 1: n).

▸ 200. [HM25] (A. Björklund, 2010.) We can use polynomial algebra, instead of backtracking, to decide whether or not a given 3D matching problem is solvable. Let the items be {a_1, ..., an}, {b_1, ..., bn}, {c_1, ..., cn}, and assign a symbolic variable to each option. If X is any subset of C, let Q(X) be the n × n matrix whose entry in row i and column j is the sum of the variables for all options ‘ai bj ck’ with ck ∉ X.

For example, suppose n = 3. The seven options t: ‘a_1 b_1 c_2’, u: ‘a_1 b_2 c_1’, v: ‘a_2 b_3 c_2’, w: ‘a_2 b_3 c_3’, x: ‘a_3 b_1 c_3’, y: ‘a_3 b_2 c_1’, z: ‘a_3 b_2 c_2’ yield the matrices

X=∅{c3}{c2}{c2,c3}{c1}{c1,c3}{c1,c2}Q(X)=(tu000v+wxy+z0)(tu000v0y+z0)(0u000wxy0)(0u00000y0)(t0000v+wxz0)(t0000v0z0)(00000wx00)

(and Q(C) is always zero). The determinant of Q(∅) is u(v+w)x − t(v+w)(y+z).

1. If the given problem has r solutions, prove that the polynomial

S=ΣX⊂C(−1)|X| det Q(X)

is the sum of r monomials, each with coefficient ±1. (In the given example it is uvx − twy.) Hint: Consider the case where all possible options are present.

2. Use this fact to design a randomized algorithm that decides q.s. whether or not a matching exists, in O(2nn^4) steps.

▸ 201. [M30] Consider the bipartite matching problem that has 3n options, ‘Xj Yk’ for 1 ≤ j, k ≤ n and (j − k)mod n ∈ {0, 1, n − 1}. (Assume that n ≥ 3.)

1. What “natural, intuitively obvious” problem is equivalent to this one?

2. How many solutions does this problem have?

3. How many updates does Algorithm X make when finding all solutions, if the items are ordered X_1, Y_1, ..., Xn, Yn, and if exercise 9 is used in step X3?

202. [13] What is

203. [M15] Equation (95) shows that the binary operation T ⊕ T′ on search trees has an identity element, ‘◼’. Is that operation (a) associative? (b) commutative?

204. [M25] True or false: Node αα′ is dominant in T ⊕ T′ if and only if α is dominant in T and α′ is dominant in T′. Hint: Express deg(αα′) in terms of deg(α) and deg(α′).

205. [M28] Prove Lemma D, about the structure of T ⊕ T′.

206. [20] If T is minimally dominant and deg(root(T)) ≤ deg(root(T′)), show that it’s easy to describe the tree T ⊕ T′.

207. [35] The principal SAT solver that we shall discuss later, Algorithm 7.2.2.2C, maintains focus by computing “activity scores,” which measure recent changes to the data structures. A similar idea can be applied to Algorithm X, by computing the score
αi=ρt1+ρt2+...,for each item  i,
where ρ (typically 0.9) is a user-specified damping factor, and where i’s list of active options was modified at times t − t_1, t − t_2, ...; here t denotes the current “time,” as measured by some convenient clock. When step X3 chooses an item for branching, the MRV heuristic of exercise 9 rates i by its degree λi = LEN(i); the new heuristic replaces this by
λi′={λi,if λi≤1;1+λi/(1+μαi),if λi≤2.

Here μ is another user-specified parameter. If μ = 0, decisions are made as before; but larger and larger values of μ cause greater and greater attention to be given to the recently active items, even if they have a somewhat large degree of branching.

1. For example, suppose αi = 1, αj = 1/2, and μ = 1. Which item will be preferable, i or j, if LEN(i) = LEN(j) + 1 and 0 ≤ LEN(j) ≤ 4?

2. What modifications to Algorithm X will implement this scheme?

3. What values of ρ and μ will avoid exponential growth, when applied to n independent copies of the toy problems (90) and (92)?

4. Does this method save time in the Y pentomino problem of Fig. 73?

▸ 208. [21] Modify the exact cover problem of Fig. 73 so that none of the Y pentominoes that occur in an ‘H’ or have been flipped over. Hint: To prevent the flipped-over Y’s marked 8 and b from occurring simultaneously, use the options ‘1c 2c 3c 4c 3b V1b’ and ‘1a 2a 3a 4a 2b V1b’, where V1b is a secondary item.

209. [20] Improve the search tree (93) in the same way that (100) improves on (91), by considering two bipairs of (92).

210. [21] A “bitriple” (α, β, γ; α′, β′, γ′) is analogous to a bipair, but with (97) replaced by α + β + γ = α′ + β′ + γ′. How can we modify an exact cover problem so that it excludes all solutions in which options α′, β′, and γ′ are simultaneously present?

211. [20] Do the options of the text’s formulation of the Langford pair problem have any bipairs? How about the n queens problem? And sudoku?

▸ 212. [M21] If the primary items of an exact cover problem have been linearly ordered, we can say that the bipair (α, β; α′, β′) is canonical if (i) the smallest item in all four options appears in α and α′; and (ii) option α is lexicographically smaller than option α′, when their items have been listed in ascending order.

1. Prove that Theorem S applies to exact coverings that are strong according to this definition of canonicity. Hint: Show that it’s a special case of the text’s definition.

2. Does such an ordering justify the choices made in (99)?

213. [M21] If π and π′ are two partitions of the same set, say that π < π′ if the restricted growth string of π is lexicographically less than the restricted growth string of π′. Let (α, β; α′, β′) be a canonical bipair in the sense of exercise 212. Also let π be a partition of the items such that α and β are two of its parts, and let π′ be the same partition but with α′ and β′ substituted. Is π < π′?

▸ 214. [21] Under the assumptions of Theorem S, how can the set of all solutions to an exact cover problem be found from the set of all strong solutions?

▸ 215. [M30] The perfect matching problem on the complete graph K2q+1 is the X2C problem with 2q+1 primary items {0, ..., 2q} and (2q+12) options ‘i j’ for 0 ≤ i < j ≤ 2q.

1. How many bipairs are present in this problem?

2. Say that (i, j, k, l) is excluded if there’s a canonical bipair (α, β; α′, β′) for which α′ = ‘i j’ and β′ = ‘k l’. Prove that, regardless of the ordering of the options, the number of excluded quadruples is 2/3 of the number of bipairs.

3. What quadruples are excluded when the options are ordered lexicographically?

4. We reduce the amount of search by introducing a secondary item (i, j, k, l) for each excluded quadruple, and appending it to the options for ‘i j’ and ‘k l’. Describe the search tree when this has been done for the quadruples of (c).

5. Show that only Θ(q^3) excluded quadruples suffice to obtain that search tree.

6. Order the options cleverly so that the search tree has only 2q + 1 nodes.

7. How many excluded quadruples suffice to obtain that search tree?

216. [25] Continuing exercise 215, experiment with the search trees that are obtained by (i) choosing a random ordering of the options, and (ii) using only m of the quadruples that are excluded by that ordering (again chosen at random).

217. [M32] A bipair of pentominoes (α, β; α′, β′) is a configuration such as

where two pentominoes occupy a 10-cell region in two different ways. In this example we may write α = S + 00 + 01 + 11 + 12 + 13, β = Y + 02 + 03 + 04 + 05 + 14, α′ = S + 04 + 05 + 12 + 13 + 14, β′ = Y + 00 + 01 + 02 + 03 + 11; hence α+β = α′ + β′ as in (97).

Compile a complete catalog of all bipairs that are possible with distinct pentominoes. In particular, show that each of the twelve pentominoes participates in at least one such bipair. (It’s difficult to do this by hand without missing anything. One good approach is to exploit the equation α − α′ = −(β − β′): First find all the delta values ±(α − α′) for each of the twelve pentominoes individually; then study all deltas that are shared by two or more of them. For example, the S and Y pentominoes both have 00 + 01 − 04 − 05 + 11 − 14 among their deltas.)

▸ 218. [20] Why must i be uncolored, in the definition of “forcing” for Algorithm P?

219. [20] Suppose p and q are primary items of an XCC problem, and that every option containing p or q includes an uncolored instance of either i or j (or both), where i and j are other items; yet p and q never occur in the same option. Prove that every option that contains i or j, but neither p nor q, can be removed without changing the problem.

220. [28] Step P5 of Algorithm P needs to emulate step C5 of Algorithm C, to see if some primary item will lose all of its options. Describe in detail what needs to be done.

221. [23] After all options that begin with item i have been examined in step P5, those that were found to be blocked appear on a stack, starting at S. Explain how to delete them. Caution: The problem might become unsolvable when an option goes away.

222. [22] Before item i is deleted in step P7, it should be removed from every option that contains S, by changing the corresponding node into a spacer. All options that involve i but not S should also be deleted. Spell out the low-level details of this process.

223. [20] Implement the output phase of Algorithm P (step P10).

▸ 224. [M21] Construct an exact cover problem with O(n) options that causes Algorithm P to perform n rounds of reduction (that is, it executes step P2 n times).

225. [21] Why does Algorithm P remove 235 options in the 6 × 10 pentomino problem, but only 151 options in the “one-sided” 6 × 15 case?

226. [M20] Assume that a_1 ... a2n is a Langford pairing, and let ak′=a2n+1−k so that a1′...a2n′ is the reverse of a_1 ... a2n. Are there any obvious relations between the sums

Σ1=Σk=12nkak,Σ1′=Σk=12nkak′Σ2=Σk=12nk2ak,Σ2′=Σk=12nk2ak′,?

What about the analogous sums S=Σk=12nkak2 and S′=Σk=12nk(ak′)2?

227. [10] What cost should be assigned to option (16), to minimize (a) ∑2? (b) S?

228. [M30] The Langford pairings for n = 16 that minimize ∑2 turn out to be precisely the 12,016 pairings that minimize ∑1; and their reversals turn out to be precisely the 12,016 pairings that maximize both ∑2 and ∑1. Is this surprising, or what?

▸ 229. [25] What Langford pairings for n = 16 are lexicographically smallest and largest?

230. [20] Explain how Algorithm X$, which minimizes the sum of option costs, can also be used to maximize that sum, in problems like that of Fig. 74.

231. [21] What’s the maximum SCRABBLE®-like score you can achieve by filling the grid below with 4-letter and 5-letter words that all are among the (a) 1000 (b) 2000 (c) 3000 most common words of English having that many letters?

For example, WATCH|AGILE|RADAR|TREND scores 26+10+7+18+14+9+5+7+24 points.

232. [20] The costs supplied to Algorithm X$ must be nonnegative integers; but d(i, j) in the 16 queens problem of Fig. 74 is never an integer. Is it OK to use $⌊d(i, j)⌋ instead of $d(i, j) for the cost of placing a queen in cell (i, j)?

233. [20] Minimize and maximize the product of the 16 queen distances, not the sum.

234. [M20] What is the minimum-cost placement of n nonattacking queens when the cost of a queen in cell (i, j) is $d(i, j)2, the square of its distance from the center?

▸ 235. [21] Solve the problem of Fig. 74 using the (integer) costs $4d(i, j)4.

▸ 236. [M41] When the cost of a queen in cell (i, j) is $d(i, j)N, for larger and larger values of N, the minimum-cost solutions to the n queens problem eventually converge to a fixed pattern. And those “ultimate” solutions turn out to be quite attractive — indeed, this family of solutions is arguably the most beautiful of all! For example, the case n = 16, illustrated here, can actually be discovered by hand, with a few moments of concentrated thought. Notice that it is doubly symmetric and nicely “rounded.”

Discover such optimum placements for as many n as you can (not by hand).

▸ 237. [M21] True or false: Two solutions to the text’s prime square problem cannot have the same product unless they are transposes of each other.

238. [24] Find 3×n arrays filled with distinct 3-digit and n-digit primes, for 3 ≤ n ≤ 7, having the minimum and maximum possible product.

▸ 239. [M27] Given a family {S_1, ..., Sm} of subsets of {1, ..., n}, together with positive weights (w_1, ..., wm), the optimum set cover problem asks for a minimum-weight way to cover {1, ..., n} with a union of Sj’s. Formulate this problem as an optimum exact cover problem, suitable for solution by Algorithm X$. Hint: Maximize the weight of all sets that do not participate in the cover.

240. [16] What usable 6-state options include MT and TX in the USA-partition problem?

241. [21] Does preprocessing by Algorithm P remove the useless option (114)?

▸ 242. [M23] Extend the algorithm of exercise 7.2.2–78 so that it visits only subgraphs that don’t cut off connected regions whose size isn’t a sum of integers in [L..U).

243. [M20] Assume that every item i of an XCC problem has been given a weight wi, and that every solution to the problem involves exactly d options. If the cost of every option is $(x^2), where x is the sum of the option’s weights, prove that every minimum-cost solution also minimizes Σk=1d(xk−r)2, for any given real number r.

244. [M21] The induced subgraphs G | U of a graph or digraph G have an interior cost, defined to be the number of ordered pairs of vertices in U that are not adjacent. For example, the interior cost of option (114) is 20, which is the maximum possible for six connected vertices of an undirected graph.

Consider any exact cover problem whose items are the vertices of G, and whose options all contain exactly t items. True or false: A solution that minimizes the sum of the interior costs also minimizes the sum of the exterior costs, as defined in the text.

245. [23] Augment the USA graph by adding a 49th vertex, DC, adjacent to MD and VA. Partition this graph into seven connected components, (a) all of size 7, removing as few edges as possible; (b) of any size, equalizing their populations as much as possible.

246. [22] The left-hand graph partition in (116) has a bizarre component that connects AZ with ND and OK, without going through NM, CO, or UT. Would we obtain more reasonable-looking solutions if we kept the same options, but minimized the exterior costs instead of the squared populations? (That is, on the left we’d consider the 34,111 options with population in [37 .. 39] million, plus two options that include New York, New England, and possibly New Jersey. The options of the right-hand example would again be the connected subsets with population in [50.5 .. 51.5] million.)

Consider also minimizing the interior costs, as defined in exercise 244.

247. [23] Specify step C1$, which takes the place of step C1 when Algorithm C is extended to Algorithm C$. Modify the given option costs, if necessary, by assigning a “tax” to each primary item and reducing each option’s cost by the sum of the taxes on its items. These new costs should be nonnegative; and every primary item should belong to at least one option whose cost is now zero. Be sure to obey condition (118).

248. [22] Let ϑ = T −Cl in step C3$, where T is the current cutoff threshold and Cl is the cost of the current partial solution on levels less than l. Explain how to choose an active item i that probably belongs to the fewest options of cost < ϑ. Instead of taking the time to make a complete search, assume conservatively that there are LEN(i) such items, after verifying that item i has at least L of them, where L is a parameter.

249. [21] A set of dk costs, with 0 ≤ c_1 ≤ c_2 ≤ ··· ≤ cdk, is said to be bad if ck + c2k + ··· + cdk ≥ θ. Design an “online algorithm” that identifies a bad set as quickly as possible, when the costs are learned one by one in arbitrary order.

For example, suppose d = 6, k = 2, and θ = 16. If costs appear in the order (3, 1, 4, 1, 5, 9, 2, 6, 5, 3, 5, 8), your algorithm should stop after seeing the 2.

250. [21] Users of Algorithm C$ are allowed to supply hints that speed up the computation, by specifying (i) a set Z of characters, such that every element of Z is the first character of exactly one primary item in every option; also (ii) a number z > 0, meaning that every option contains exactly z primary items whose names don’t begin with a character in Z. (For example, Z = {p, r, c, b} in the sudoku options (30); z = 1 in options (110). In the options (16) for Langford pairs, we could change the name of each numeric item i to ‘!i’, then let Z = {!} and z = 2.) Explain how to use these hints to supply an early-cutoff test at the beginning of step C3$, as explained in the text.

251. [18] If a given problem is solvable, when does Algorithm Z first discover that fact?

▸ 252. [20] Algorithm Z produces the ZDD (120) from the options (121) if step Z3 simply chooses the leftmost item i_1 = RLINK(0) instead of using the MRV heuristic. What ZDD would have been obtained if the method of exercise 9 had been used instead?

▸ 253. [21] Extend Algorithm Z so that it reports the total number of solutions.

▸ 254. [28] The signature σ computed by Algorithm Z in step Z2 is supposed to characterize the current subproblem completely. It contains one bit for each primary item, indicating whether or not that item still needs to be covered.

1. Explain why one bit isn’t sufficient for secondary items with colors.

2. Suggest a good way to implement the computation of σ.

3. Algorithm C uses the operations hide′ and unhide′ in (50)–(57), in order to avoid unnecessary accesses to memory in nodes for secondary items. Explain why Algorithm Z does not want to use those optimizations. Hint: Algorithm Z needs to know whether the option list for a secondary item is empty.

4. When the list for item i is purified, its options of the wrong color are removed from other lists. But they remain on list i, in order to be unpurified later. How then can Algorithm Z know when list i is no longer relevant to the current subproblem?

255. [HM29] Express the exact number of updates made by Algorithm Z when it finds the perfect matchings of KN , as well as the exact number of ZDD nodes produced, in terms of Fibonacci numbers. Hint: See exercise 193.

▸ 256. [M23] What is the behavior of Algorithm Z when it is asked to find all perfect matchings of the “bizarre” graph (89)?

▸ 257. [21] How does Algorithm Z do on the “extreme” exact cover problem, with n items and 2n − 1 options? (See the discussion preceding (82).)

1. What signatures are formed in step Z2?

2. Draw the schematic ZDD, analogous to (123), when n = 4.

258. [HM21] How many updates does Algorithm Z perform, in that extreme problem?

259. [M25] Exercise 196 analyzes the behavior of Algorithm X on the bounded permutation problem defined by a_1 ... an. Show that Algorithm Z is considerably faster, by determining the number of memos, ZDD nodes, and updates when a_1a_2 ... an−1an is (a) n n...n n [with n! solutions]; (b) 2 3 ... n n [with 2n−1 solutions]. Assume that the items are X_1, X_2, ..., Xn, Y_1, Y_2, ..., Yn, in that order.

260. [M21] Exercises 14 and 201 are bipartite matching problems related to choosing seats at a circular table. Test Algorithm Z empirically on those problems, and show that it solves the latter in linear time (despite exponentially many solutions).

▸ 261. [23] Let G be a directed acyclic graph, with source vertices S and sink vertices T .

1. Use Algorithm C (or Z) to find all sets of m vertex-disjoint paths from S to T .

2. Also find all such sets of paths from sk to tk for 1 ≤ k ≤ m, given sk and tk.

3. Apply (a) to find all sets of n − 1 disjoint paths that enter an n × n square at the north or east edge, proceed by south and/or west steps, and exit at the south or west edge, avoiding the corners. (A random 16 × 16 example is shown.)

4. Apply (b) to find all vertex-disjoint, downward paths of eight knights that start on the top row of a chessboard and end on the bottom row in reverse order.

▸ 262. [M23] One of the advantages of Algorithm Z is that a ZDD allows us to generate uniformly random solutions. (See the remarks following 7.1.4–(13).)

1. Determine the number of ZDD nodes output by Algorithm Z for the set of all domino tilings of Sn, where Sn is the shape obtained after right triangles of side 7 have been removed from each corner of a 16 × n rectangle:

How many tilings are possible for S_16 (the Aztec diamond of order 8)? For S_32?

2. Similarly, determine the ZDD size for the family of all diamond tilings of Tn — the grid simplex (n + 16, n + 8, 16, n + 8, 0, 0, 0), a hexagon of sides (8, 8, n, 8, 8, n):

263. [24] Compare the time and space requirements of Algorithms C and Z when they are applied to (a) the 16 queens problem; (b) pentominoes, as in exercises 271 and 274; (c) MacMahon’s triangle problem, as in exercise 126; (d) the generalized de Bruijn sequences of exercise 95; (e) the “right word stair” problem of exercise 90; (f) the 6 × 6 “word search” problem of exercise 105; (g) the kakuro problem in exercise 431.

264. [M21] Suppose step Z3 always chooses the first active item i = RLINK(0), instead of using the MRV heuristic, unless some other active item has LEN(i) = 0. Prove that Algorithm Z will then output an ordered ZDD.

▸ 265. [22] Prove that Algorithm Z will never produce identical ZDD nodes (o¯i?li:hi)=(o¯j?lj:hj) for i ≠ j, if all items are primary. But secondary items can cause duplicates.

Exercises—Second Set

Thousands of fascinating recreational problems have been based on polyominoes and their polyform cousins (the polycubes, polyiamonds, polyhexes, polysticks, ... ). The following exercises explore “the cream of the crop” of such classic puzzles, as well as a few gems that were not discovered until recently.

In most cases the point of the exercise is to find a good way to discover all solutions, usually by setting up an appropriate exact cover problem that can be solved without taking an enormous amount of time.

▸ 266. [25] Sketch the design of a utility program that will create sets of options by which an exact cover solver will fill a given shape with a given set of polyominoes.

267. [18] Using Conway’s piece names, pack five pentominoes into the shape so that they spell a common English word when read from left to right.

▸ 268. [21] There are 1010 ways to pack the twelve pentominoes into a 5 × 12 box, not counting reflections. What’s a good way to find them all, using Algorithm X?

269. [21] How many of those 1010 packings decompose into 5 × k and 5 × (12 − k)?

270. [21] In how many ways can the eleven nonstraight pentominoes be packed into a 5 × 11 box, not counting reflections as different? (Reduce symmetry cleverly.)

271. [20] There are 2339 ways to pack the twelve pentominoes into a 6 × 10 box, not counting reflections. What’s a good way to find them all, using Algorithm X?

272. [23] Continuing exercise 271, explain how to find special kinds of packings:

1. Those that decompose into 6 × k and 6 × (10 − k).

2. Those that have all twelve pentominoes touching the outer boundary.

3. Those with all pentominoes touching that boundary except for V, which doesn’t.

4. Same as (c), with each of the other eleven pentominoes in place of V.

5. Those with the minimum number of pentominoes touching the outer boundary.

6. Those that are characterized by Arthur C. Clarke’s description, as quoted below.

That is, the X pentomino should touch only the F (aka R), the N (aka S), the U, and the V — no others.

Very gently, he replaced the titanite cross
in its setting between the F, N, U, and V pentominoes.

— ARTHUR C. CLARKE, Imperial Earth (1976)

273. [25] All twelve pentominoes fit into a 3 × 20 box only in two ways, shown in (3^6).

1. How many ways are there to fit eleven of them into that box?

2. In how many solutions to (a) are the five holes nonadjacent, kingwise?

3. In how many ways can eleven pentominoes be packed into a 3 × 19 box?

274. [21] There are five different tetrominoes, namely

In how many essentially different ways can each of them be packed into an 8 × 8 square together with the twelve pentominoes?

275. [21] If an 8 × 8 checkerboard is cut up into thirteen pieces, representing the twelve pentominoes together with one of the tetrominoes, some of the pentominoes will have more black cells than white. Is it possible to do this in such a way that U, V, W, X, Y, Z have a black majority while the others do not?

276. [18] Design a nice, simple tiling pattern that’s based on the five tetrominoes.

277. [25] How many of the 6 × 10 pentomino packings are strongly three-colorable, in the sense that each individual piece could be colored red, white, or blue in such a way that no pentominoes of the same color touch each other — not even at corner points?

▸ 278. [32] Use the catalog of bipairs in exercise 217 to reduce the number of 6 × 10 pentomino packings, listing strong solutions only (see Theorem S). How much time is saved?

279. [40] (H. D. Benjamin, 1948.) Show that the twelve pentominoes can be wrapped around a cube of size 10×10×10. For example, here are front and back views of such a cube, made from twelve colorful fabrics by the author’s wife in 1993:

(Photos by Héctor García)

What is the best way to do this, minimizing undesirable distortions at the corners?

▸ 280. [M26] Arrange the twelve pentominoes into a Möbius strip of width 4. The pattern should be “faultfree”: Every straight line must intersect some piece.

▸ 281. [20] The white cells of a (2n+1)×(2n+1) checkerboard, with black corners, form an interesting graph called the Aztec diamond of order n; and the black cells form the Aztec diamond of order n+1/2. For example, the diamonds of orders 11/2 and 13/2 are

except that (ii) has a “hole” of order 3/2. Thus (i) has 61 cells, and (ii) has 80.

1. Find all ways to pack (i) with the twelve pentominoes and one monomino.

2. Find all ways to pack (ii) with the 12 pentominoes and 5 tetrominoes.

Speed up the process by not producing solutions that are symmetric to each other.

▸ 282. [22] (Craig S. Kaplan.) A polyomino can sometimes be surrounded by non-overlapping copies of itself that form a fence: Every cell that touches the polyomino — even at a corner — is part of the fence; conversely, every piece of the fence touches the inner polyomino. Furthermore, the pieces must not enclose any unoccupied “holes.”

Find the (a) smallest and (b) largest fences for each of the twelve pentominoes. (Some of these patterns are unique, and quite pretty.)

283. [22] Solve exercise 282 for fences that satisfy the tatami condition of exercise 7.1.4–215: No four edges of the tiles should come together at any “crossroads.”

▸ 284. [27] Solomon Golomb discovered in 1965 that there’s only one placement of two pentominoes in a 5 × 5 square that blocks the placement of all the others.

Place (a) {O, P, U, V} and (b) {P, R, T, U} into a 7 × 7 square in such a way that none of the other eight will fit in the remaining spaces.

285. [21] (T. H. O’Beirne, 1961.) The one-sided pentominoes are the eighteen distinct 5-cell pieces that can arise if we aren’t allowed to flip pieces over:

Notice that there now are two versions of P, Q, R, S, Y, and Z.

In how many ways can all eighteen of them be packed into rectangles?

286. [21] If you want to pack the twelve pentominoes into a 6 × 10 box without turning any pieces over, 2^6 different problems arise, depending on the orientations of the one-sided pieces. Which of those 64 problems has (a) the fewest (b) the most solutions?

▸ 287. [23] A princess asks you to pack an m × n box with pentominoes, rewarding you with $c · (ni + j) if you’ve covered cell (i, j) with piece c, where c = (1, 2, ..., 12) for pieces (O, P, ..., Z). (The most valuable packing will be “closest to alphabetic order.”)

Use Algorithm X$ to maximize your bounty when packing boxes of sizes 4 × 15, 5 × 12, 6 × 10, 10 × 6, 12 × 5, and 15 × 4. Consider also the princess’s circle-shaped subset of a 9 × 9 box, where you are to cover only the 60 cells whose distances from the center are between 1 and 18. How do the running times of Algorithm X$ compare to the amounts of time that Algorithm X would take to find all solutions?

288. [21] Similarly, pack the one-sided pentominoes optimally into 9 × 10 and 10 × 9.

▸ 289. [29] (Pentominoes of pentominoes.) Magnify the 3 × 20 pentomino packing (3^6) by replacing each of its unit cells by (a) 3 × 4 rectangles; (b) 4 × 3 rectangles. In how many ways can the resulting 720-cell shape be packed with twelve complete sets of twelve pentominoes, using one set for each of the original pentomino regions?

(c) Also partition the 720-cell shape below into 3 × 20 approximately square 12-cell regions, by assigning each gray cell to an adjacent region. (This shape has been superimposed on a grid whose 12×12 regions are perfectly square.) Minimize the total perimeter of the 60 resulting regions, and try for a pleasantly symmetrical solution.

Use your partition to present a scaled-up version of (3^6), again with 12 complete sets.

290. [21] When tetrominoes are both checkered and one-sided (see exercises 275 and 285), ten possible pieces arise. In how many ways can all ten of them fill a rectangle?

291. [24] (A puzzle a day.) Using the two trominoes, the five tetrominoes, and three of the pentominoes, one can cover up 11 of the 12 “months” and 30 of the 31 “days” in the following pair of diagrams, thereby revealing the current month and day:

Which of the (123) sets of three pentominoes always allow this to be done?

292. [20] There are 35 hexominoes, first enumerated in 1934 by the master puzzlist H. D. Benjamin. At Christmastime that year, he offered ten shillings to the first person who could pack them into a 14 × 15 rectangle — although he wasn’t sure whether or not it could be done. The prize was won by F. Kadner, but not as expected: Kadner proved that the hexominoes actually can’t be packed into any rectangle! Nevertheless, Benjamin continued to play with them, eventually discovering that they fit nicely into the triangle shown here.

Prove Kadner’s theorem. Hint: See exercise 275.

293. [24] (Frans Hansson, 1947.) The fact that 35 = 1^2 + 3^2 + 5^2 suggests that we might be able to pack the hexominoes into three boxes that represent a single hexomino shape at three levels of magnification, such as

For which hexominoes can this be done?

▸ 294. [30] Show that the 35 hexominoes can be packed into five “castles”:

In how many ways can this be done?

295. [41] For which values of m can the hexominoes be packed into a box like this?

296. [41] Perhaps the nicest hexomino packing uses a 5 × 45 rectangle with 15 holes

proposed by W. Stead in 1954. In how many ways can the 35 hexominoes fill it?

297. [24] (P. Torbijn, 1989.) Can the 35 hexominoes be packed into six 6 × 6 squares?

▸ 298. [22] In how many ways can the twelve pentominoes be placed into an 8 × 10 rectangle, leaving holes in the shapes of the five tetrominoes? (The holes should not touch the boundary, nor should they touch each other, even at corners; one example is shown at the right.) Explain how to encode this puzzle as an XCC problem.

299. [39] If possible, solve the analog of exercise 298 for the case of 35 hexominoes in a 5 × 54 rectangle, leaving holes in the shapes of the twelve pentominoes.

▸ 300. [23] In how many ways can the twelve pentominoes be arranged in a 10 × 10 square, filling exactly six of the cells in every row and exactly six of the cells in every column, if we also require that (a) the cells on both diagonals are completely empty? (b) the cells on both diagonals are completely filled? (c) the design is really interesting?

301. [25] Here’s one way to place the twelve pentominoes into a 5 × 5 square, covering the cells of rows (1, 2, 3, 4, 5) exactly (2, 3, 2, 3, 2) times:

1. How many such placements are possible?

2. Suppose we’ve placed O first, P next, Q next, ..., Z last, when making the arrangement above. Then Z is above W is above V is above U is above P is above O; hence the pentominoes have been stacked up on six levels. Show that a different order of placement would require only four levels.

3. Find a solution to (a) that needs only three levels.

4. Find a solution to (a) that can’t be achieved with only four levels.

302. [26] Say that an n-omino is “small” if it fits in a (n+1)×(n+1) box, and “slim” if it contains no 2 × 2 tetromino. Thus, for example, pentominoes O, Q, S, Y aren’t small; P isn’t slim.

1. How many nonominoes are both small and slim?

2. Fit nine different small-and-slim nonominoes into a 9 × 9 box.

3. Use a solution to (b) as the basis of a jigsaw sudoku puzzle with a unique solution. The clues of your puzzle should be the initial digits of π.

▸ 303. [HM35] A parallelogram polyomino, or “parallomino” for short, is a polyomino whose boundary consists of two paths that each travel only north and/or east. (Equivalently, it is a “staircase polygon,” “skew Young tableau,” or a “skew Ferrers board,” the difference between the diagrams of two tableaux or partitions; see Sections 5.1.4 and 7.2.1.4.) For example, there are five parallominoes whose boundary paths have length 4:

1. Find a one-to-one correspondence that maps the set of ordered trees with m leaves and n nodes into the set of parallominoes with width m and height n − m. The area of each parallomino should be the path length of its corresponding tree.

2. Study the generating function G(w,x,y)=Σparallominoes wareaxwidthyheight.

3. Prove that the parallominoes whose width-plus-height is n have total area 4n−2.

4. Part (c) suggests that we might be able to pack all of those parallominoes into a 2n−2 × 2n−2 square, without rotating them or flipping them over. Such a packing is clearly impossible when n = 3 or n = 4; but is it possible when n = 5 or n = 6?

304. [M25] Prove that it’s NP-complete to decide whether or not n given polyominoes, each of which fits in a Θ(log n) × Θ(log n) square, can be exactly packed into a square.

305. [25] When a square grid is scaled by 1/2 and rotated 45°, we can place half of its vertices on top of the original ones; the other “odd-parity” vertices then correspond to the centers of the original square cells.

Using this idea we can glue a small domino of area 1 over portions of an ordinary domino of area 2, thereby obtaining ten distinct two-layer pieces called the windmill dominoes:

1. Arrange four windmill dominoes so that the upper layer resembles a windmill.

2. Place all ten windmill dominoes inside a 4 × 5 box, without overlapping.

3. Similarly, pack them all into a 2 × 10 box.

4. Place them so that the upper layer fills a (4/2)×(5/2) rectangle.

5. Similarly, fit the upper layer into a (2/2)×(10/2) rectangle.

In each case (a)–(e), use Algorithm X to count the total number of possible placements. Also look at the output and choose arrangements that are especially pleasing.

▸ 306. [30] (S. Grabarchuk, 1996.) In how many ways can the ten windmill dominoes be arranged so that the 20 large squares define a snake-in-the-box cycle, in the sense of exercise 172(b), and so do the 20 small squares? (For example, arrangements like

satisfy one snake-in-the-box condition but not the other.)

307. [M21] If a (3m+1)×(3n+2) box is packed with 3mn+2m+n straight trominoes and one domino, where must the domino be placed?

308. [22] A polyiamond is a connected set of triangles in a triangular grid, inspired by the diamond — just as a polyomino is a connected set of squares in a square grid, inspired by the domino . Thus we can speak of moniamonds, diamonds, triamonds, etc.

1. Extend exercise 266 to the triangular grid, using the coordinate system of exercise 124. How many base placements do each of the tetriamonds have?

2. Find all ways to pack the pentiamonds into a convex polygon (see exercise 143).

3. Similarly, find all such ways to pack the one-sided pentiamonds.

309. [24] The hexiamonds are particularly appealing, because — like pentominoes — there are 12 of them. Here they are, with letter names suggested by J. H. Conway:

1. How many base placements do they have?

2. In how many ways can they be packed into convex polygons, as in exercise 308?

310. [23] What’s the smallest m for which the 12 hexiamonds fit without overlap in

Find a pleasant way to place them inside of that smallest box.

▸ 311. [30] (Hexiamond wallpaper.) Place the twelve hexiamonds into a region of N triangles, so that (i) shifted copies of the region fill the plane; (ii) the hexiamonds of the resulting infinite pattern do not touch each other, even at vertices; (iii) N is minimum.

312. [22] The following shape can be folded, to cover the faces of an octahedron:

Fill it with hexiamonds so that they cross the folded edges as little as possible.

▸ 313. [29] (Hexiamonds of hexiamonds.) A “whirl,” shown here, is an interesting dodeciamond that tiles the plane in a remarkably beautiful way.

If each triangle ‘Δ’ of a hexiamond is replaced by a whirl, in how many ways can the resulting 72-triangle shape be packed with the full set of hexiamonds? (Exercise 289 discusses the analogous problem for pentominoes.)

Consider also using “flipped whirls,” the left-right reflections of each whirl.

▸ 314. [28] (G. Sicherman, 2008.) Can the four pentiamonds be used to make two 10-iamonds of the same shape? Formulate this question as an exact cover problem.

315. [20] A polyhex is a connected shape formed by pasting hexagons together at their edges, just as polyominoes are made from squares and polyiamonds are made from triangles. For example, there’s one monohex and one dihex, but there are three trihexes. Chemists have studied polyhexes since the 19th century, and named the small ones:

(Groups of six carbon atoms can bond together in a nearly planar fashion, forming long chains of hexagons, with hydrogen atoms attached. But the correspondence between polyhexes and polycyclic aromatic hydrocarbons is not exact.)

Represent the individual hexagons of an infinite grid by Cartesian-like coordinates

where 1¯=−1,2¯=−2, etc. Extending exercises 266 and 308(a), explain how to find the base placements of a polyhex, given the coordinates of its cells when placed on this grid.

316. [20] Show that the complete set of trihexes and tetrahexes can be packed nicely into a rosette that consists of 37 concentric hexagons. In how many ways can it be done?

317. [22] (Tetrahexes of tetrahexes.) If we replace each hexagon of a tetrahex by a rosette of seven hexagons, we get a 28-hex. In how many ways can that scaled-up shape be packed with the seven distinct tetrahexes? (See exercises 289 and 313.)

▸ 318. [20] Let’s say that the T-grid is the set of all hexagons xy with x ≢ y (modulo 3):

Show that there’s a one-to-one correspondence between the hexagons of the T-grid and the triangles of the infinite triangular grid, in which every polyiamond corresponds to a polyhex. (Therefore the study of polyiamonds is a special case of the study of polyhexes!) Hint: Exercise 124 discusses a coordinate system for representing triangles.

319. [21] After polyominoes, polyiamonds, and polyhexes, the next most popular polyforms are the polyaboloes, originally proposed by S. J. Collins in 1961. These are the shapes obtainable by attaching isosceles right triangles at their edges; for example, there are three diaboloes: Notice that any n-abolo corresponds to a 2n-abolo, when it has been scaled up by 2.

The 14 tetraboloes can be named by using rough resemblances to hexiamonds:

Show that the study of polyaboloes can be reduced to the study of (slightly generalized) polyominoes, just as exercise 318 reduces polyiamonds to polyhexes.

▸ 320. [M28] Explain how to enumerate all of the N-aboloes that are convex. How many of the convex 56-aboloes can be packed by the fourteen tetraboloes?

321. [24] (T. H. O’Beirne, 1962.) In how many ways can a square be formed from the eight one-sided tetraboloes and their mirror images?

322. [23] The polysticks provide us with another intriguing family of shapes that can be combined in interesting ways. An “n-stick” is formed by joining n horizontal and/or vertical unit line segments together near grid points. For example, there are two disticks and five tristicks; and of course there’s only one monostick. They’re shown here in white, surrounded by the sixteen tetrasticks in black.

Polysticks introduce yet another twist into polyform puzzles, because we must not allow different pieces to cross each other when we pack them into a container. Extend exercise 266 to polysticks, so that Algorithm X can deal with them conveniently.

323. [M25] We’ve now seen polyominoes, polyiamonds, polyhexes, ..., polysticks, each of which have contributed new insights; and many other families of “polyforms” have in fact been studied. Let’s close our survey with polyskews, a relatively new family that seems worthy of further exploration. Polyskews are the shapes that arise when we join squares alternately with rhombuses, in checkerboard fashion. For example, here are the ten tetraskews:

There are two monoskews, one diskew, and five triskews.

1. Explain how to draw such skewed pixel diagrams.

2. Show that polyskews, like polyaboloes, can be reduced to polyominoes.

3. In how many ways do the tetraskews make a skewed rectangle?

▸ 324. [20] Extend exercise 266 to three dimensions. How many base placements do each of the seven Soma pieces have?

325. [27] The Somap is the graph whose vertices are the 240 distinct solutions to the Soma cube problem, with u — v if and only if u can be obtained from an equivalent of v by changing the positions of at most three pieces. The strong Somap is similar, but it has u — v only when a change of just two pieces gets from one to the other.

1. What are the degree sequences of the Somap graphs?

2. How many connected components do they have? How many bicomponents?

▸ 326. [M25] Use factorization to prove that Fig. 75’s W-wall cannot be built.

327. [24] Figure 75(a) shows some of the many “low-rise” (2-level) shapes that can be built from the seven Soma pieces. Which of them is hardest (has the fewest solutions)? Which is easiest? Answer those questions also for the 3-level prism shapes in Fig. 75(b).

▸ 328. [M23] Generalizing the first four examples of Fig. 75, study the set of all shapes obtainable by deleting three cubies from a 3 × 5 × 2 box. (Two examples are shown here.) How many essentially different shapes are possible? Which shape is easiest? Which shape is hardest?

329. [22] Similarly, consider (a) all shapes that consist of a 3 × 4 × 3 box with just three cubies in the top level; (b) all 3-level prisms that fit into a 3 × 4 × 3 box.

330. [25] How many of the 1285 nonominoes define a prism that can be realized by the Soma pieces? Do any of those packing problems have a unique solution?

331. [M40] Make empirical tests of Piet Hein’s belief that the number of shapes achievable with seven Soma pieces is approximately the number of 27-cubie polycubes.

332. [20] (B. L. Schwartz, 1969.) Show that the Soma pieces can make shapes that appear to have more than 27 cubies, because of holes hidden inside or at the bottom:

In how many ways can each of these three trick shapes be constructed?

333. [22] Show that the seven Soma pieces can also make structures such as

which are “self-supporting” via gravity. (You may need to place a small book on top.)

Fig. 75. Gallery of noteworthy polycubes that contain 27 cubies. All of them can be built from the seven Soma pieces, except for the W-wall. Many constructions are also stable when tipped on edge and/or when turned upside down. (See exercises 326–334.)

▸ 334. [M32] Impossible structures can be built, if we insist only that they look genuine when viewed from the front (like façades in Hollywood movies)! Find all solutions to

that are visually correct. (To solve this exercise, you need to know that the illustrations here use the non-isometric projection (x, y, z) ↦ (30x − 42y, 14x + 10y + 45z)u from three dimensions to two, where u is a scale factor.) All seven Soma pieces must be used.

335. [30] The earliest known example of a polycube puzzle is the “Cube Diabolique,” manufactured in late nineteenth century France by Charles Watilliaux; it contains six flat pieces of sizes 2, 3, ..., 7:

1. In how many ways do these pieces make a 3 × 3 × 3 cube?

2. Are there six polycubes, of sizes 2, 3, ..., 7, that make a cube in just one way?

336. [21] (The L-bert Hall.) Take two cubies and drill three holes through each of them; then glue them together and attach a solid cubie and dowel, as shown. Prove that there’s only one way to pack nine such pieces into a 3 × 3 × 3 box.

337. [29] (Angus Lavery, 1989.) Design a puzzle that consists of nine bent tricubes, whose face squares are either blank or colored with a red or green spot. The goal is to assemble the pieces into a 3 × 3 × 3 cube in two ways:

1. No green spots are visible, and the red spots match a left-handed die.

2. No red spots are visible, and the green spots match a right-handed die.

338. [22] Show that there are exactly eight different tetracubes — polycubes of size 4. Which of the following shapes can they make, respecting gravity? How many solutions are possible?

339. [25] How many of the 369 octominoes define a 4-level prism that can be realized by the tetracubes? Do any of those packing problems have a unique solution?

340. [30] There are 29 pentacubes, conveniently identified with one-letter codes:

Pieces o through z are called, not surprisingly, the solid pentominoes or flat pentacubes.

1. What are the mirror images of a, b, c, d, e, f, A, B, C, D, E, F, j, k, l, ..., z?

2. In how many ways can the solid pentominoes be packed into an a × b × c cuboid?

3. What “natural” set of 25 pentacubes is able to fill the 5 × 5 × 5 cube?

▸ 341. [25] The full set of 29 pentacubes can build an enormous variety of elegant structures, including a particularly stunning example called “Dowler’s Box.” This 7 × 7 × 5 container, first considered by R. W. M. Dowler in 1979, is constructed from five flat slabs. Yet only 12 of the pentacubes lie flat; the other 17 must somehow be worked into the edges and corners.

Despite these difficulties, Dowler’s Box has so many solutions that we can actually impose many further conditions on its construction:

1. Build Dowler’s Box in such a way that the chiral pieces a, b, c, d, e, f and their images A, B, C, D, E, F all appear in horizontally mirror-symmetric positions.

2. Alternatively, build it so that those pairs are diagonally mirror-symmetric.

3. Alternatively, place piece x in the center, and build the remaining structure from four congruent pieces that have seven pentacubes each.

342. [25] The 29 pentacubes can also be used to make the shape shown here, exploiting the curious fact that 3^4 +4^3 = 29 · 5. But Algorithm X will take a long, long time before telling us how to construct it, unless we’re lucky, because the space of possibilities is huge. How can we find a solution quickly?

343. [40] (T. Sillke, 1995.) For each of the twelve pentomino shapes, build the tallest possible tower whose walls are vertical and whose floors all have the given shape, using distinct pentacubes. Hint: Judicious factorization will give tremendous speedup.

344. [20] In how many distinct ways can a 5 × 5 × 5 cube by packed with 25 solid Y pentominoes? (See Fig. 73.) Discuss how to remove the 48 symmetries of this problem.

345. [20] Pack twelve U-shaped dodecacubes into a 4 × 6 × 6 box without letting any two of them form a “cross.”

346. [M30] An (l, m, n)-tripod is a cluster of l + m + n + 1 cubies in which three “legs” of lengths l, m, and n are attached to a corner cubie, as in the (1,2,3)-tripod shown here. A “pod” is the special case where the tripod is (l, m, n) ∪ {(l′, m, n) | 0 ≤ l′ < l} ∪ {(l, m′, n) | 0 ≤ m′ < m} ∪ {(l, m, n′) | 0 ≤ n′ < n}.

1. Prove that, for all m, n ≥ 0, shifted copies of nonoverlapping (1, m,n)-tripods are able to fill all of 3-dimensional space, without rotation or reflection. Hint: Pack N^2 of them into an N × N × N torus, where N = m + n +2.

2. Show that 7/9 of 3-dimensional space can be packed with shifted (2, 2, 2)-tripods.

3. Similarly, at least 65/108 of 3D space can be packed with shifted (3, 3, 3)-tripods.

4. Let r(l, m, n) be the maximum number of pods that can be packed in an l × m×n cuboid. Prove that at least (1+l+m+n)r(l, m, n)/(4lmn) of 3-dimensional space can be packed with shifted (l, m, n)-tripods.

5. Use Algorithm M to evaluate r(l, m, n) for 4 ≤ l ≤ m ≤ n ≤ 6.

▸ 347. [M21] (N. G. de Bruijn, 1961.) Prove that an l × m × n box can be completely filled with 1 × 1 × k bricks only if k is a divisor of l, m, or n. (Consequently, it can be completely filled with a × b × c bricks only if a, b, and c all satisfy this condition.)

348. [M41] Find the maximum number of “canonical bricks” (1 × 2 × 4) that can be packed into an l × m × n box, leaving as few empty cells as possible.

▸ 349. [M27] (D. Hoffman.) Show that 27 bricks of size a × b × c can always be packed into an s × s × s cube, where s = a + b + c. But if s/4 < a < b < c, 28 bricks won’t fit.

350. [22] Can 28 bricks of size 3 × 4 × 5 be packed into a 12 × 12 × 12 cube?

351. [M46] Can 5^5 hypercuboids of size a × b × c × d × e always be packed into a 5-dimensional hypercube of size (a + b + c + d + e) ··· (a + b + c + d + e)?

352. [21] In how many ways can the 12 pentominoes be packed into a 2 × 2×3 × 5 box?

353. [20] A weak polycube is a set of cubies that are loosely connected via common edges, not necessarily via common faces. In other words, we consider cubies to be adjacent when their centers are at most 2 units apart; up to 18 neighbors are possible. Find all the weak polycubes of size 3, and pack them into a symmetrical container.

▸ 354. [M30] A polysphere is a connected set of spherical cells that belong to the “face-centered cubic lattice,” which is one of the two principal ways to pack cannonballs (or oranges) with maximum efficiency. That lattice is conveniently regarded as the set S of all quadruples (w, x, y, z) of integers for which w + x + y + z = 0. Each cell of S has 12 neighbors, obtained by adding 1 to one coordinate and subtracting 1 from another.

It’s instructive to view S in two different ways, by slicing it into plane layers that either have constant x+y+z (hence constant w) or constant y+z (hence constant w+x):

(Here stands for (w, x, y, z).) If we include the layers above and below, we get

with each sphere nestling in the gap between the three or four spheres below it. In the “hex layers” on the left, (w, x, y, z) lies directly above (w +3, x − 1, y − 1, z − 1), but doesn’t touch it; in the “quad layers” on the right, (w, x, y, z) lies directly above (w +1, x +1, y − 1, z − 1), but doesn’t touch it.

1. Show that every polyomino, and every polyhex, may be regarded as a polysphere:

2. Conversely, every planar polysphere looks like either a polyomino or a polyhex.

3. Every polysphere {(w_1, x_1, y_1, z_1), ..., (wn, xn, yn, zn)} has a unique base placement {(w1′,x1′,y1′,z1′),...,(wn′,xn′,yn′,zn′)} obtained by subtracting (w′, x′, y′, z′) from each (wk, xk, yk, zk), where x′ = min{x_1, ..., xn}, y′ = min{y_1, ..., yn}, z′ = min{z_1, ..., zn}, and w′ + x′ + y′ + z′ = 0. Prove that xk′+yk′+zk′<n.

4. As with polycubes, we say that polyspheres v and v′ are equivalent if the base placement of v is also a base placement of some rotation of v′ in three dimensions. (Reflections of “chiral” polyspheres are not considered to be equivalent.) Formally speaking, a rotation of S about a line through the origin is an orthogonal 4 × 4 matrix that has determinant 1 and preserves w + x + y + z. Find such matrices for (i) rotation of the hex layers by 120°; (ii) rotation of the quad layers by 90°.

5. A planar polysphere is equivalent to its reflection, because we can rotate by 180° around a line in its plane. Find suitable 4 × 4 matrices by which we can legally reflect polyspheres that are equivalent to (i) polyominoes; (ii) polyhexes.

6. Prove that every rotation that takes a polysphere into another polysphere is obtainable as a product of the matrices exhibited in (d) and (e).

355. [25] The theory in exercise 354 allows us to represent polysphere cells with three integer coordinates xyz, because x, y, and z are nonnegative in base placements. The other variable, w, is redundant (but worth keeping in mind); it always equals −x−y−z.

1. What’s a good way to find all the base placements of a given polysphere {x_1y_1z_1, x_2y_2z_2, ..., xnynzn}? Hint: Use exercise 354 to tweak the method of exercise 324.

2. Any three points of three-dimensional space lie in a plane. So exercise 354(b) tells us that there are just four trispheres: a tromino, two trihexes, and one that’s both:

What are their base placements?

3. According to exercise 354(c), every base placement of a tetrasphere occurs in the SGB graph simplex (3, 3, 3, 3, 3, 0, 0). Use exercise 7.2.2–75 to find all of the four-element connected subsets of that graph, and identify all of the distinct tetraspheres. How many times does each tetrasphere occur in the graph?

356. [23] Polysphere puzzles often involve the construction of three kinds of shapes:

(An n × n roof or stretched roof is called an “n-pyramid” or a “stretched n-pyramid.”)

1. Define each of these configurations by specifying a suitable base placement.

2. Each of the shapes illustrated is made from 20 spheres, and so is the stretched 4 × 3 roof. Find all multisets of five tetraspheres that suffice to make these shapes.

3. The 4-pyramid and the stretched 4-pyramid involve 30 spheres. What multisets of ten trispheres are able to make them?

4. The truncated octahedron, which represents all permutations of {1, 2, 3, 4}, is a noteworthy 24-cell subset of S (see exercise 5.1.1– 10). What multisets of six tetraspheres can build it?

357. [M40] Investigate “polysplatts,” which are the sets of truncated octahedra that can be built by pasting adjacent faces together (either square or hexagonal).

358. [HM41] Investigate “polyhexaspheres,” which are the connected sets of spheres in the hexagonal close packing. (This packing differs from that of exercise 354 because each sphere of a hexagonal layer is directly above a sphere that’s 2, not 3, layers below it.)

359. [29] Nick Baxter devised an innocuous-looking but maddeningly difficult “Square Dissection” puzzle for the International Puzzle Party in 2014, asking that the nine pieces

be placed flat into a 65 × 65 square. One quickly checks that 17 × 20+18 × 20+···+24 × 25 = 65^2; yet nothing seems to work! Solve his puzzle with the help of Algorithm X.

▸ 360. [20] The next group of exercises is devoted to the decomposition of rectangles into rectangles, as in the Mondrianesque pattern shown here. The reduction of such a pattern is obtained by distorting it, if necessary, so that it fits into an m × n grid, with each of the vertical coordinates {0, 1, ..., m} used in at least one horizontal boundary and each of the horizontal coordinates {0, 1, ..., n} used in at least one vertical boundary. For example, the illustrated pattern reduces to , where m = 3 and n = 5. (Notice that the original rectangles needn’t have rational width or height.)

A pattern is called reduced if it is equal to its own reduction. Design an exact cover problem by which Algorithm M will discover all of the reduced decompositions of an m × n rectangle, given m and n. How many of them are possible when (m, n) = (3, 5)?

361. [M25] The maximum number of subrectangles in a reduced m × n pattern is obviously mn. What is the minimum number?

362. [10] A reduced pattern is called strictly reduced if each of its subrectangles [a..b)×[c..d) has (a, b) ≠ (0, m) and (c, d) ≠ (0, n) — in other words, if no subrectangle “cuts all the way across.” Modify the construction of exercise 360 so that it produces only strictly reduced solutions. How many 3 × 5 patterns are strictly reduced?

363. [20] A rectangle decomposition is called faultfree if it cannot be split into two or more rectangles. For example, is not faultfree, because it has a fault line between rows 2 and 3. (It’s easy to see that every reduced faultfree pattern is strictly reduced, unless m = n = 1.) Modify the construction of exercise 360 so that it produces only faultfree solutions. How many reduced 3 × 5 patterns are faultfree?

364. [23] True or false: Every faultfree packing of an m×n rectangle by 1 × 3 trominoes is reduced, except in the trivial cases (m, n) = (1, 3) or (3, 1).

365. [22] (Motley dissections.) Many of the most interesting decompositions of an m×n rectangle involve strictly reduced patterns whose subrectangles [ai .. bi)×[ci .. di) satisfy the extra condition

(ai,bi)≠(aj,bj)and(ci,di)≠(cj,dj)when i≠j.

Thus no two subrectangles are cut off by the same pair of horizontal or vertical lines. The smallest such “motley dissections” are the 3 × 3 pinwheels, and , which are considered to be essentially the same because they are mirror images of each other. There are eight essentially distinct motley rectangles of size 4 × n, namely

The two 4 × 4s can each be drawn in 8 different ways, under rotations and reflections. Similarly, most of the 4 × 5s can be drawn in 4 different ways. But the last two have only two forms, because they’re symmetric under 180° rotation. (And the last two are actually equivalent, if we swap the two x coordinates in the middle.)

Design an exact cover problem by which Algorithm M will discover all of the motley dissections of an m × n rectangle, given m and n. (When m = n = 4 the algorithm should find 8 + 8 solutions; when m = 4 and n = 5 it should find 4 + 4 + 4 + 4 + 2 + 2.)

▸ 366. [25] Improve the construction of the previous exercise by taking advantage of symmetry to cut the number of solutions in half. (When m = 4 there will now be 4 + 4 solutions when n = 4, and 2+2+2+2+1+1 when n = 5.) Hint: A motley dissection is never identical to its left-right reflection, so we needn’t visit both.

367. [20] The order of a motley dissection is the number of subrectangles it has. There are no motley dissections of order six. Show, however, that there are m × m motley dissections of order 2m−1 and m×(m+1) motley dissections of order 2m, for all m > 3.

368. [M21] (H. Postl, 2017.) Show that an m × n motley dissection of order t can exist only if n < 2t/3. Hint: Consider adjacent subrectangles.

369. [21] An m × n motley dissection must have order less than (m+12), because only (m+12)−1 intervals [ai .. bi) are permitted. What is the maximum order that’s actually achievable by an m × n motley dissection, for m = 5, 6, and 7?

▸ 370. [23] Explain how to generate all of the m × n motley dissections that have 180° rotational symmetry, as in the last two examples of exercise 365, by modifying the construction of exercise 366. (In other words, if [a..b) × [c..d) is a subrectangle of the dissection, its complement [m − b..m − a) × [n − d..n − c) must also be one of the subrectangles, possibly the same one.) How many such dissections have size 8 × 16?

371. [24] Further symmetry is possible when m = n (as in exercise 365’s pinwheel).

1. Explain how to generate all of the n×n motley dissections that have 90°-rotational symmetry. This means that [a..b) × [c..d) implies [c..d) × [n−b..n−a).

2. Explain how to generate all of the n × n motley dissections that are symmetric under reflection about both diagonals. This means that [a..b) × [c..d) implies [c..d)×[a..b) and [n−d..n−c)×[n−b..n−a), hence [n−b..n−a)×[n−d..n−c).

3. What’s the smallest n for which symmetric solutions of type (b) exist?

▸ 372. [M35] (Floorplans.) If a rectangle decomposition satisfies the tatami condition — “no four rectangles meet” — it’s often called a floorplan, and its subrectangles are called rooms. The line segments that delimit rooms are called bounds. Four possibilities arise when room r is adjacent to bound s: Either s ↓ r, r → s, r ↓ s, or s → r, meaning respectively that the top, right, bottom, or left boundary of r is part of s.

For example, the floorplans shown on the next page have 10 rooms {A, B, ..., J}, 7 + 6 bounds {h_0, ..., h_6, v_0, ..., v_5}, and the following adjacencies: h_0 ↓ A ↓ h_3 ↓ D ↓ h_5 ↓ E ↓ h_6, h_0 ↓ B ↓ h_1 ↓ C ↓ h_3 ↓ F ↓ h_6, h_1 ↓ G ↓ h_2 ↓ H ↓ h_4 ↓ I ↓ h_6, h_2 ↓ J ↓ h_6; v_0 → A → v_1 → B → v_5, v_1 → C → v_3 → H → v_4, v_0 → D → v_2 → F → v_3 → G → v_5, v_0 → E → v_2, v_3 → I → v_4 → J → v_5.

Two floorplans with the same adjacencies are considered to be equivalent. Thus, all four of the floorplans above are essentially the same, even though they look rather different: In particular, room C needn’t overlap room D; we require only C ↓ h_3 ↓ D.

1. Let r ⇓ r′ mean that r = r_0 ↓ s_0 ↓ r_1 ↓ ··· ↓ sk−1 ↓ rk = r′ for some k > 0; define r ⇒ r′ similarly. Prove that [r ⇓ r′] + [r ⇒ r′] + [r′ ⇓ r] + [r′ ⇒ r] = 1, when r ≠ r′. Hint: Every floorplan has unique diagonal and antidiagonal equivalents, as shown.

2. A twin tree is a data structure whose nodes v have four pointer fields, L0(v), R0(v), L1(v), R1(v). It defines two binary trees T_0 and T_1 on the nodes, where Tθ is rooted at ROOTθ and has child links (Lθ, Rθ). These two trees satisfy inorder(T_0) = inorder(T_1)= v_1 ... vn; R0(vk) = ∧ ⇔ R1(vk) ≠ ∧, for 1 ≤ k < n.

For each room r, if r’s top left corner is a ⊤ junction, set L0(r) ← ∧ and L1(r) ← r′, where r′ is the room opposite r in that corner; otherwise reverse the roles of L0 and L1. Similarly, set R0(r) ← ∧ and R1(r) ← r′ if the bottom right corner of r is a ⊣ junction, or vice versa otherwise. (Use r′ = ∧ at extreme corners.) Also set ROOT0 and ROOT1 to the bottom-left and top-right rooms. Show that a twin tree is created, convenient for representing this floorplan.

373. [26] A “perfectly decomposed rectangle” of order t is a faultfree dissection of a rectangle into t subrectangles [ai .. bi) × [ci .. di) such that the 2t dimensions b_1 − a_1, d_1 − c_1, ..., bt − at, dt − ct are distinct. For example, five rectangles of sizes 1 × 2, 3 × 7, 4 × 6, 5 × 10, and 8 × 9 can be assembled to make the perfectly decomposed 13 × 13 square shown here. What are the smallest possible perfectly decomposed squares of orders 5, 6, 7, 8, 9, and 10, having integer dimensions?

374. [M28] An “incomparable dissection” of order t is a decomposition of a rectangle into t subrectangles, none of which will fit inside another. In other words, if the heights and widths of the subrectangles are respectively h_1 × w_1, ..., ht × wt, we have neither (hi ≤ hj and wi ≤ wj) nor (hi ≤ wj and wi ≤ hj) when i ≠ j.

1. True or false: An incomparable dissection is perfectly decomposed.

2. True or false: The reduction of an incomparable dissection is motley.

3. True or false: The reduction of an incomparable dissection can’t be a pinwheel.

4. Prove that every incomparable dissection of order ≤ 7 reduces to the first 4 × 4 motley dissection in exercise 365; and its seven regions can be labeled as shown, with h_7 < h_6 < ··· < h_2 < h_1 and w_1 < w_2 < ··· < w_6 < w_7.

5. Suppose the reduction of an incomparable dissection is m × n, and suppose its regions have been labeled {1, ..., t}. Then there are numbers x_1, ..., xn, y_1, ..., ym such that the widths are sums of the x’s and the heights are sums of the y’s. (For example, in (d) we have w_2 = x_1, h_2 = y_1 + y_2 + y_3, w_7 = x_2 + x_3 + x_4, h_7 = y_1, etc.) Prove that such a dissection exists with w_1 < w_2 < ··· < wt if and only if the linear inequalities w_1 < w_2 < ··· < wt have a positive solution (x_1, ..., xn) and the linear inequalities h_1 > h_2 > ··· > ht have a positive solution (y_1, ..., ym).

375. [M29] Among all the incomparable dissections of order (a) seven and (b) eight, restricted to integer sizes, find the rectangles with the smallest possible semiperimeter (height plus width). Also find the smallest possible squares that have incomparable dissections in integers. Hint: Show that there are 2t potential ways to mix the h’s with the w’s, preserving their order; and find the smallest semiperimeter for each of those cases.

▸ 376. [M25] Find seven different rectangles of area 1/7 that can be assembled into a square of area 1, and prove that the answer is unique.

377. [M28] Two rectangles of shapes h × w and h′ × w′ can be concatenated to form a larger rectangle of size (h + h′) × w if w = w′, or of size h × (w + w′) if h = h′.

1. Given a set S of rectangle shapes, let ∧(S) be the set of all shapes that can be made from the elements of S by repeated concatenation. Describe ∧({1 × 2, 3 × 1}).

2. Find the smallest S ⊆ T such that T ⊆ ∧(S), where T = {h×w | 1 < h < w}.

3. What’s the smallest S with ∧(S) = { h×w | h, w > 1 and hw mod 8 = 0}?

4. Given m and n, solve (c) with ∧(S) = { h×w | h, w > m and hw mod n = 0}.

▸ 378. [M30] (A finite basis theorem.) Continuing exercise 377, prove that any set T of rectangular shapes contains a finite subset S such that T ⊆ ∧(S).

▸ 379. [23] What h×w rectangles can be packed with copies of the Q pentomino? Hint: It suffices to find a finite basis for all such rectangles, using the previous exercise.

380. [35] Solve exercise 379 for the Y pentomino.

381. [20] Show that 3n copies of the disconnected shape can pack a 12 × n rectangle for all sufficiently large values of n.

▸ 382. [18] There’s a natural way to extend the idea of motley dissection to three dimensions, by subdividing an l × m × n cuboid into subcuboids [ai .. bi) × [ci .. di) × [ei .. fi) that have no repeated intervals [ai .. bi) or [ci .. di) or [ei .. fi).

For example, Scott Kim has discovered a remarkable motley 7 × 7 × 7 cube consisting of 23 individual blocks, 11 of which are illustrated here. (Two of them are hidden behind the others.) The full cube is obtained by suitably placing a mirror image of these pieces in front, together with a 1 × 1 × 1 cubie in the center.

By studying this picture, show that Kim’s construction can be defined by coordinate intervals [ai .. bi)×[ci .. di)×[ei .. fi), with 0 ≤ ai, bi, ci, di, ei, fi ≤ 7 for 1 ≤ i ≤ 23, in such a way that the pattern is symmetrical under the transformation xyz⟼y¯z¯x¯. In other words, if [a..b) × [c..d) × [e..f) is one of the subcuboids, so is [7 − d .. 7 − c) × [7 − f .. 7 − e) × [7 − b .. 7 − a).

383. [29] Use exercise 382 to construct a perfectly decomposed 92 × 92 × 92 cube, consisting of 23 subcuboids that have 69 distinct integer dimensions. (See exercise 373.)

384. [24] By generalizing exercises 365 and 366, explain how to find every motley dissection of an l × m × n cuboid, using Algorithm M. Note: In three dimensions, the strictness condition ‘(ai,bi) ≠ (0, m) and (ci, di) ≠ (0, n)’ of exercise 362 should become

[(ai,bi)=(0,l)]+[(ci,di)=(0,m)]+[(ei,fi)=(0,n)]≤1.

What are the results when l = m = n = 7?

385. [M36] (H. Postl, 2017.) Arbitrarily large motley cuboids can be constructed by repeatedly nesting one motley cuboid within another (see answer 367). Say that a motley cuboid is primitive if it doesn’t contain a nested motley subcuboid.

Do primitive motley cuboids of size l × m × n exist only when l = m = n = 7?

▸ 386. [M34] A polyomino can have eight different types of symmetry:

(Case (i) is often called 8-fold symmetry; case (iii) is often called central symmetry; case (vi) is often called left-right symmetry. Cases (ii), (iv), (v) are 4-fold symmetries; cases (ii) and (iii) are rotation symmetries; cases (iv)–(vii) are reflection symmetries.) In each case an n-omino of that symmetry type has been shown, where n is minimum.

How many symmetry types can a polyiamond or polyhex have? Give example n-iamonds and n-hexes of each type, where n is minimum.

▸ 387. [M36] Continuing exercise 386, how many symmetry types can a polycube have? Give an example of each type, using the minimum number of cubies. (Note that mirror reflection is not a legal symmetry for a polycube; L-twist ≠ R-twist!)

Exercises—Third Set

The following exercises are based on several intriguing logic puzzles that have recently become popular: futoshiki, kenken, masyu, slitherlink, kakuro, etc. Like sudoku, these puzzles typically involve a hidden pattern, for which only partial information has been revealed. The point of each exercise is usually to set up an appropriate exact cover problem, and to use it either to solve such a puzzle or to create new ones.

▸ 388. [21] The goal of a futoshiki puzzle is to deduce the entries of a secret latin square, given only two kinds of hints: A “strong clue” is an explicit entry; a “weak clue” is a greater-than relation between neighboring entries. The entries are the numbers 1 to n, where n is usually 5 as in the following examples:

Solve these puzzles by hand, using sudoku-like principles.

389. [20] Sketch a simple algorithm that finds simple lower and upper bounds for each entry that is part of a weak clue in a futoshiki puzzle, by repeatedly using the rule that a ≤ x < y ≤ b implies x ≤ b − 1 and y ≥ a + 1. (Your algorithm shouldn’t attempt to give the best possible bounds; that would solve the puzzle! But it should deduce the values of five entries in puzzle (a) of exercise 388, as well as entry (4, 2) of puzzle (b).)

▸ 390. [21] Show that every futoshiki puzzle is a special case of an exact cover problem. In fact, show that every such puzzle can be formulated in at least two different ways:

1. Use a pairwise ordering trick analogous to (25) or (26), to encode the weak clues.

2. Use color controls to formulate an XCC problem suitable for Algorithm C.

391. [20] A futoshiki puzzle is said to be valid if it has exactly one solution. Use Algorithm X to generate all possible 5 × 5 latin squares. Explain why many of them can’t be the solution to a valid futoshiki puzzle unless it has at least one strong clue.

▸ 392. [25] There are 26(406)=245656320 ways to construct a 5 × 5 futoshiki puzzle that has six weak clues and no strong ones. How many of them (a) are valid? (b) have no solutions? (c) have more than one solution? Also refine those counts, by considering how many such puzzles of types (a), (b), and (c) have at least one “long path” p < q < r < s < t (like the path that’s present in exercise 388(a)). Give an example of each case.

393. [25] There are 56(256)=2767187500 ways to construct a 5 × 5 futoshiki puzzle that has six strong clues and no weak ones. How many of them (a) are valid? (b) have no solutions? (c) have more than one solution? Give an example of each case.

394. [29] Show that every 5 × 5 futoshiki puzzle that has only five clues — strong, weak, or a mixture of both — has at least four solutions. Which puzzles attain this minimum?

395. [25] Continuing exercise 391, find a 5 × 5 latin square that cannot be the solution to a valid futoshiki puzzle unless at least three strong clues have been given.

▸ 396. [25] Inspired by exercise 388(c), construct a valid 9 × 9 futoshiki puzzle whose diagonal contains the strong clues (3, 1, 4, 1, 5, 9, 2, 6, 5) in that order. Every other clue should be a weak ‘<’—not a ‘>’, not a ‘∧’, not a ‘∨’.

▸ 397. [30] (Save the sheep.) Given a grid in which some of the cells are occupied by sheep, the object of this puzzle is to construct a fence that keeps all the sheep on one side. The fence must begin and end at the edge of the grid, and it must follow the grid lines without visiting any point twice. Furthermore, exactly two edges of each sheep’s square should be part of the fence. For example, consider the following 5 × 5 grids:

The four sheep on the left can be “saved” only with the fence shown in the middle. Once you understand why, you’ll be ready to save the four sheep on the right.

1. Explain how Algorithm C can help to solve puzzles like this, by showing that every solution satisfies a certain XCC problem. Hint: Imagine “coloring” each square with 0 or 1, with 1 indicating the cells on the sheep’s side of the fence.

2. Devise an interesting 8 × 8 puzzle that has a unique solution and at most 10 sheep.

398. [23] (KenKen®.) A secret latin square whose entries are {1, 2, ..., n} can often be deduced by means of arithmetic. A kenken puzzle specifies the sum, difference, product, or quotient of the entries in each of its “cages,” which are groups of cells indicated by heavy lines, as in the following examples:

(When the operation is ‘−’ or ‘÷’, the cage must have just two cells. A one-cell cage simply states its contents, without any operation; hence its solution is a no-brainer.)

Cages look rather like the boxes of jigsaw sudoku (see (34)); but in fact the rules are quite different: Two entries of the same cage can be equal, if they belong to different rows and different columns. For example, the ‘9×’ in (a) can be achieved only by multiplying the three entries {1, 3, 3}; hence there’s exactly one way to fill that cage.

Solve (a), (b), (c) by hand. Show that one of them is actually not a valid puzzle.

▸ 399. [22] How can all solutions to a kenken puzzle be obtained with Algorithm C?

400. [21] Many clues of a kenken puzzle often turn out to be redundant, in the sense that the contents of one cage might be fully determined by the clues from other cages. For example, it turns out that any one of the clues in puzzle 398(a) could actually be omitted, without permitting a new solution.

Find all subsets of those 11 clues that suffice to determine a unique latin square.

401. [22] Find all 4 × 4 kenken puzzles whose unique solution is the latin square shown at the right, and whose cages all have two cells. Furthermore, there should be exactly two cages for each of the four operations +, −, ×, ÷.

1234214343123421

402. [24] Solve this 12 × 12 kenken puzzle, using hexadecimal digits from 1 to C:

The five-cell cages of this puzzle have multiplicative clues, associated with the names of the twelve pentominoes:

O,9240×P,5184×Q,3168×R,720×S,15840×T,19800×U,10560×V,4032×W,1620×X,5040×Y,576×Z,17248×

▸ 403. [31] Inspired by exercises 398(a) and 398(c), construct a valid 9 × 9 kenken puzzle whose clues exactly match the decimal digits of π, for as many places as you can.

▸ 404. [25] (Hidato®.) A “hidato solution” is an m × n matrix whose entries are a permutation of {1, 2, ..., mn} for which the cells containing k and k + 1 are next to each other, either horizontally, vertically, or diagonally, for 1 ≤ k < mn. (In other words, it specifies a Hamiltonian path of king moves on an m × n board.) A “hidato puzzle” is a subset of those numbers, which uniquely determines the others; the solver is supposed to recreate the entire path from the given clues.

For example, consider the 4 × 4 puzzle (i). There’s only one place to put ‘2’. Then there are two choices for ‘4’; but one of them blocks the upper left corner (see (ii)), so we must choose the other. Similarly, ‘6’ must not block any corner. Therefore (iii) is forced; and it’s easy to fill in all of the remaining blanks, thereby obtaining solution (iv).

Explain how to encode such puzzles for solution by Algorithm C.

405. [21] The preceding exercise needs a subroutine to determine the endpoints of all simple paths of lengths 1, 2, ..., L from a given vertex v in a given graph. That problem is NP-hard; but sketch an algorithm that works well for small L in small graphs.

406. [16] Show that the following hidato puzzle isn’t as hard as it might look at first:

▸ 407. [20] Here’s a curious 4 × 8 array that is consistent with 52 hidato solutions:

Change it to a valid hidato puzzle, by adding one more clue.

408. [28] (N. Beluhov.) Construct 6 × 6 hidato puzzles that have (a) only five clues; (b) at least eighteen clues, all of which are necessary.

▸ 409. [30] Can the first 10 clues of a 10 × 10 hidato puzzle be the first 20 digits of π?

410. [22] (Slitherlink.) Another addictive class of puzzles is based on finding closed paths or “loops” in a given graph, when the allowable cycles must satisfy certain constraints. For instance, a slitherlink puzzle prescribes the number of loop edges that surround particular cells of a rectangular grid, as in diagram (i) below.

The first step in solving puzzle (i) is to note where the secret edges are definitely absent or definitely present. The 0s prohibit not only the edges immediately next to them but also a few more, because the path can’t enter a dead end. Conversely, the 3 forces the path to go through the upper left corner; we arrive at situation (ii):

Some experimentation now tells us which edge must go with the lower 1. We must not form two loops, as in (iii) or (iv). And hurrah: There’s a unique solution, (v).

Which of the following 5 × 5 slitherlink diagrams are valid puzzles? Solve them.

411. [20] True or false: A slitherlink diagram with a numeric clue given in every cell always has at most one solution. Hint: Consider the 2 × 2 case.

▸ 412. [22] A “weak solution” to a slitherlink diagram is a set of edges that obeys the numeric constraints, and touches every vertex of the grid either twice or not at all; but it may form arbitrarily many loops. For example, the diagram of exercise 410(i) has six weak solutions, three of which are shown in 410(iii), (iv), and (v).

Show that there’s a nice way to obtain all the weak solutions of a given diagram, by formulating a suitable XCC problem. Hint: Think of the edges as constructed from tiles centered at the vertices, and use even/odd coordinates as in answer 133.

▸ 413. [30] Explain how to modify Algorithm C so that the construction of exercise 412 will produce only the true “single-loop” solutions. Your modified algorithm shouldn’t be specific to slitherlink; it should apply also to masyu and other loop-discovery puzzles.

414. [25] The “strongest possible” answer to exercise 413 would cause the modified Algorithm C to backtrack as soon as the current choice of edge colors is incompatible with any single loop. Show that the algorithm in that answer is not as strong as possible, by examining its behavior on the puzzle at the right.

▸ 415. [M33] Exactly 5·(2^25 −1) nonempty slitherlink diagrams of size 5 × 5 are “homogeneous,” in the sense that all of their clues involve the same digit d ∈ {0, 1, 2, 3, 4}. (See exercise 410(a)–(d).) How many of them are valid puzzles? What are the minimum and maximum number of clues, for each d, in puzzles that contain no redundant clues?

416. [M30] For each d ∈ {0, 1, 2, 3, 4}, construct valid n×n slitherlink diagrams whose nonblank clues are all equal to d, for infinitely many n.

417. [M46] (N. Beluhov, 2018.) Exercise 410(a, b, d) illustrates three homogeneous slitherlink puzzles that are valid for exactly the same pattern of nonblank clues. Do infinitely many such square puzzles exist?

418. [M29] An m × n slitherlink diagram is said to be symmetrical if cells (i, j) and (m − 1 − i, n − 1 − j) are both blank or both nonblank, for 0 ≤ i < m and 0 ≤ j < n. (Many grid-based puzzles obey this oft-unwritten rule.)

1. There are exactly 6^25 ≈ 2.8 × 10^19 slitherlink diagrams of size 5 × 5, since each of the 25 cells can contain either ‘0’, ‘1’, ‘2’, ‘3’, ‘4’, or ‘ ’. How many of those diagrams are symmetrical?

2. How many of the symmetric diagrams in (a) are valid puzzles?

3. How many of those valid puzzles are minimal, in the sense that the deletion of nonblank clues in (i, j) and (4 − i, 4 − j) would make the solution nonunique?

4. What is the minimum number of clues in a valid 5 × 5 symmetrical puzzle?

5. What is the maximum number of clues in a minimal 5 × 5 symmetrical puzzle?

419. [30] What surprise is concealed in the following symmetrical slitherlink puzzle?

420. [M22] Consider an m × n slitherlink with m and n odd, having 2s in the pattern

(and possibly other clues). Show that there’s no solution if m mod 4 = n mod 4 = 1.

▸ 421. [20] (Masyu.) A masyu (“evil influence”) puzzle, like slitherlink, conceals a hidden loop of straight segments. But there are two important differences. First, the loop passes through the centers of grid cells, instead of following the edges. Second, no numerical quantities are involved; the clues are entirely visual and geometrical.

Clues appear in circles through which the loop must pass: (i) The path must turn 90° at every black circle; but it must travel straight through the two neighboring cells just before and after turning. (ii) The path must not turn 90° when it goes through a white circle; and it must not travel straight through the two neighboring cells just before and after not turning. (Thus it must actually turn, at one or both of those cells. We get at least one turn per clue, and at least one straight segment.)

Consider, for example, a 5 × 5 puzzle with a black clue in cell 02, and with white clues in cells 13, 30, 32, and 43 as shown. The loop clearly will have to include the subpaths 20 — 30 — 40 — 41 and 42 — 43 — 44 — 34 in some order. It also must include either 00 — 01 — 02 — 12 — 22 or 04 — 03 — 02 — 12 — 22, because of the black clue. But the latter alternative is impossible, because it leaves no way to go straight through the white clue in 13. Thus 10 — 00 — 01 — 02 — 12 — 22 is forced; and also 23 — 13 — 03 — 04 — 14 — 24 — 34. (We couldn’t go 24 — 23, because that would close the loop prematurely.) The rest of the path now sort of falls into place.

Show that one of the clues in this example puzzle is actually redundant. But if any of the other four clues are absent, show that alternative solutions are possible.

422. [21] Show that the “weak solutions” to any given masyu puzzle are the solutions to an easily constructed XCC problem, by adapting the solution of exercise 412.

▸ 423. [M25] For each of the (m−1)n + m(n−1) potential edges e in the solution of an m × n masyu puzzle, let xe be the Boolean variable ‘[e is present]’. The XCC problem constructed in exercise 422 is essentially a set of constraints on those variables.

Explain how to improve that construction dramatically, by exploiting the following special property that is enjoyed by masyu puzzles: Let N, S, E, and W be the edges leading out of a cell that holds a clue. If the clue is black, we have N = ~S and E = ~W ; if the clue is white, we have N = S, E = W , and E = ~N. (Thus every clue reduces the number of independent variables by at least 2.)

▸ 424. [36] Make an exhaustive study of 6 × 6 masyu, and gather whatever statistics you think are particularly interesting. For example, how many of the 3^36 ≈ 1.5 × 10^17 ways to place white or black clues lead to a valid puzzle? Which of the valid puzzles have the fewest clues? the most clues? the shortest loops? the longest loops? only white clues? only black clues? How many of those puzzles are minimal, in the sense that none of their clues can be removed without allowing a new solution?

How many of the 2^36 ≈ 6.9 × 10^10 ways to occupy cells occur as the pattern of white clues in a valid puzzle? How many of them occur as the pattern of black clues? How many puzzles remain valid when white and black are interchanged? Which 6 × 6 masyu puzzle do you think is most difficult to solve?

425. [28] The solution to a masyu puzzle is composed of five kinds of “tiles”: and blank. For example, the 3 × 3 solution shown here contains two tiles of each nonblank type.

Find 4 × 4, 5 × 5, and 6 × 6 puzzles whose unique solutions have exactly k tiles of each nonblank type, for every possible k.

▸ 426. [31] Obtain a valid masyu puzzle from diagram (i) below by changing each ‘’ clue into either ‘’ or ‘’.

▸ 427. [25] Design a 25 × 25 masyu puzzle by adding white clues (only) to diagram (ii) above. All of your clues should preserve the 8-fold symmetry of this pattern.

428. [M28] For infinitely many n, construct a valid n×n masyu puzzle with O(n) clues whose loop goes through all four corner cells, where all clues are (a) black; (b) white.

429. [21] A closed path on a triangular grid may have “sharp turns,” which change the direction by 120°, or “slack turns,” which change the direction by 60°, or both. Therefore triangular masyu has three flavors of clues: ‘’ for the sharp turns, ‘’ for the slack turns, and of course ‘’ for the non-turns.

1. Solve the following homogeneous triangular masyu puzzles:

2. The following patterns for triangular masyu are clearly impossible to solve. But show that each of them is solvable if the colors {, , } are suitably permuted:

▸ 430. [26] (Kakuro.) A kakuro puzzle is like a crossword puzzle, except that its “words” are blocks of two or more nonzero digits {1, 2, ..., 9}, not strings of letters. The digits of each block must be distinct, and their sum is given as a clue. Every cell to be filled belongs to exactly one horizontal block and one vertical block.

For example, the mini-kakuro shown here has just three horizontal blocks and three vertical blocks. Notice that the desired sums are indicated to the immediate left or above each block; thus the first horizontal block is supposed to be filled with two digits that sum to 5, so there are four possibilities: 14, 23, 32, 41. The first vertical block should sum to 6; again there are four possibilities, this time 15, 24, 42, 51 (because 33 is forbidden). The second horizontal block has three digits that should sum to 19; it is considerably less constrained. Indeed, there are thirty ways to obtain 19-in-three, namely the permutations of {2, 8, 9} or {3, 7, 9} or {4, 6, 9} or {4, 7, 8} or {5, 6, 8}.

1. Solve the puzzle. Hint: There’s only one possibility for the lower right corner.

2. Sketch a simple way to build a table of all suitable combinations of n-in-k, for 2 ≤ k ≤ 9 and 2 ≤ n ≤ 45. Which n and k have the most? Hint: Use bitmaps.

3. Generalized kakuro is a related puzzle, for which each block of length k has a specified set of combinations, chosen from among the (9k) possibilities (regardless of their sum). For example, suppose the three horizontal blocks of mini-kakuro must be filled respectively with permutations of {1, 3}, {3, 5}, or {5, 7}; {1, 3, 5}, {1, 7, 9}, {2, 4, 6}, {6, 8, 9}, or {7, 8, 9}; {2, 4}, {4, 6}, or {6, 8}; and require the same for the three vertical blocks. Find the unique solution to that puzzle.

4. It would be easy to formulate kakuro as an XCC problem, as we did word squares in exercise 87, by simply giving one option for each possible placement of a block. But the resulting problem might be gigantic: For example, long blocks are not uncommon in kakuro, and each 9-digit block would have 9! = 362,880 options(!). Show that generalized kakuro can be formulated efficiently as an XCC problem.

▸ 431. [30] The inventor of kakuro, Jacob E. Funk of Manitoba (who always called his puzzles “Cross Sums”), published the following challenge on pages 50 and 66 of the August/September 1950 issue of Dell Official Crossword Puzzles:

Many ingenious constructions are present here; but unfortunately, he failed to realize that there is more than one solution. Find all solutions, and obtain a valid puzzle by repairing some of his original clues.

▸ 432. [M25] We can’t simply design new kakuro puzzles by randomly filling the blanks and using the resulting sums as the constraints, because the vast majority of feasible sums yield nonunique solutions. Verify this experimentally for the generic diagrams a)

In each case determine the exact number of ways to fill the blanks, without repeated digits in any row or column, as well as exactly how many of those filled-in diagrams are uniquely reconstructible from their block sums. Consider also symmetry.

433. [26] Six of the sum-clues in this little kakuro diagram are unspecified:

In how many ways can you obtain valid puzzles by specifying them?

434. [30] Exactly how many kakuro diagrams are possible in a 9 × 9 grid? (Every row and every column should contain at least one block of empty cells, except that the topmost row and leftmost column are completely black. All blocks must have length ≥ 2. Empty cells needn’t be rookwise connected.) What is the maximum number of blocks?

435. [31] Design a rectangular kakuro puzzle for which the blocks at the top of the solution are 31, 41, 59, 26, 53, 58, 97 (the first fourteen digits of π).

▸ 436. [20] (Hitori.) Let’s wind up this potpourri of examples by considering a completely different combinatorial challenge. A hitori puzzle (“alone”) is an m × n array in which we’re supposed to cross elements out until three conditions are achieved:

1. No row or column contains repeated elements.

2. Adjacent elements cannot be crossed out.

3. The remaining elements are rookwise connected.

For example, consider the 4 × 5 word rectangle (α). Conditions (i) and (ii) can be satisfied in sixteen ways, such as (β) and (γ). But only (δ) satisfies also (iii).

A crossed-out cell is said to be black; the other cells are white. While solving a hitori, it’s helpful to circle an entry that is certain to become white. We can initially circle all the “seeds” — the entries that don’t match any others in their row or column.

For example, puzzle (α) has eight seeds. If we decide to blacken a cell, we immediately circle its neighbors (because they cannot also be black). Thus, for instance, we shouldn’t cross out the E in cell (2, 4): That would circle the L in (2, 3), forcing the other L to be black and cutting off the corner E as in (β).

The precise value of a seed is immaterial to the puzzle; it can be replaced by any other symbol that differs from everything else in its row or column.

We say as usual that a hitori puzzle is valid if it has exactly one solution. Explain why (a) a valid hitori puzzle has exactly one solution with all seeds white; (b) a hitori puzzle that has a unique solution with all seeds white is valid if and only if all the seed cells not adjacent to black in that solution are “articulation points” for the set of white cells — that is, their removal would disconnect the whites. (See (3, 1) and (3, 2) in (δ).)

▸ 437. [21] A weak solution to a hitori puzzle is a solution for which all seeds are white, and for which properties (i) and (ii) of exercise 436 hold. Given a hitori puzzle, define an XCC problem whose solutions are precisely its weak solutions.

438. [30] Explain how to modify Algorithm C so that, when given an XCC problem from the construction in answer 437, it will produce only solutions that satisfy also the connectivity condition (iii). Hint: See exercise 413; also consider reachability.

439. [M20] Let G be a graph on the vertices V . A hitori cover of G is a set U ⊆ V such that (i) G | U is connected; (ii) if v/∈ U and u — v then u ∈ U; (iii) if u ∈ U and if v ∈ U for all u — v, then G | (U \ u) is not connected.

1. Describe a hitori cover in terms of standard graph theory terminology.

2. Show that the solution of a valid hitori puzzle is a hitori cover of Pm □ Pn.

440. [21] True or false: If the letter A occurs exactly twice in the top row of a valid hitori puzzle, exactly one of those occurrences will survive in the solution.

441. [18] Describe every valid hitori puzzle of size 1 × n on a d-letter alphabet.

▸ 442. [M33] Enumerate all hitori covers of Pm □ Pn, for 1 ≤ m ≤ n ≤ 9.

▸ 443. [M30] Prove that an m × n hitori cover has at most (mn + 2)/3 black cells.

444. [M27] Can a valid n × n hitori puzzle involve fewer than 2n/3 distinct elements? Construct a valid puzzle of size 3k × 3k, using only the elements {0, 1, ..., 2k}.

▸ 445. [M22] It’s surprisingly difficult to construct a valid hitori puzzle that has no seeds. In fact, there are no n × n examples for n ≤ 9 except when n = 6. But it turns out that quite a few seedless 6 × 6 hitori puzzles do exist.

Consider the five hitori covers below. Determine, for each of them, the exact number of valid hitori puzzles with no seeds, having that pattern of white and black cells as the solution. Hint: In some cases the answer is zero.

▸ 446. [24] The digits of e, 2.718281828459045..., are well known to have a curious repeating pattern. In fact, the first 25 digits actually define a valid 5 × 5 hitori puzzle! What is the probability that a random 5 × 5 array of decimal digits will have that property? And what about octal digits? Hexadecimal digits?

447. [22] (Johan de Ruiter.) Are there any values of m > 1 and n > 1 for which the first mn digits of π define a valid m × n hitori puzzle?

448. [22] Do any of the 31344 double word squares formed from WORDS(3000) make valid hitori puzzles? (See exercise 87.)

449. [40] (Hidden nuggets.) Johan de Ruiter noticed in 2017 that George Orwell had included a valid hitori puzzle in his novel Nineteen Eighty-Four (part 2, chapter 9):

Did Homer, Shakespeare, Tolstoy, and others also create hitori puzzles accidentally?

450. [22] Use Algorithm X to solve the “tot tibi sunt dotes” problem of Section 7.2.1.7.

We should “play up” the role of play.

— FRANCIS EDWARD SU, “Mathematics for Human Flourishing” (2017)

He reaps no satisfaction but from low and sensual objects,
or from the indulgence of malignant passions.

— DAVID HUME, The Sceptic (1742)

I can’t get no . . .

— MICK JAGGER and KEITH RICHARDS, Satisfaction (1965)

7.2.2.2. Satisfiability

We turn now to one of the most fundamental problems of computer science: Given a Boolean formula F (x_1, ..., xn), expressed in so-called “conjunctive normal form” as an AND of ORs, can we “satisfy” F by assigning values to its variables in such a way that F(x_1, ..., xn) = 1? For example, the formula

F(x1,x2,x3)=(x1∨x¯2)∧(x2∨x3)∧(x¯1∨x¯3)∧(x¯1∨x¯2∨x¯3)(1)

is satisfied when x_1x_2x_3 = 001. But if we rule that solution out, by defining

G(x1,x2,x3)=F(x1,x2,x3)∧(x1∨x2∨x¯3),(2)

then G is unsatisfiable: It has no satisfying assignment.

Section 7.1.1 discussed the embarrassing fact that nobody has ever been able to come up with an efficient algorithm to solve the general satisfiability problem, in the sense that the satisfiability of any given formula of size N could be decided in NO(1) steps. Indeed, the famous unsolved question “does P = NP?” is equivalent to asking whether such an algorithm exists. We will see in Section 7.9 that satisfiability is a natural progenitor of every NP-complete problem.*

* At the present time very few people believe that P = NP [see SIGACT News 43, 2 (June 2012), 53–77]. In other words, almost everybody who has studied the subject thinks that satisfiability cannot be decided in polynomial time. The author of this book, however, suspects that NO(1)-step algorithms do exist, yet that they’re unknowable. Almost all polynomial time algorithms are so complicated that they lie beyond human comprehension, and could never be programmed for an actual computer in the real world. Existence is different from embodiment.

On the other hand enormous technical breakthroughs in recent years have led to amazingly good ways to approach the satisfiability problem. We now have algorithms that are much more efficient than anyone had dared to believe possible before the year 2000. These so-called “SAT solvers” are able to handle industrial-strength problems, involving millions of variables, with relative ease, and they’ve had a profound impact on many areas of research and development such as computer-aided verification. In this section we shall study the principles that underlie modern SAT-solving procedures.

To begin, let’s define the problem carefully and simplify the notation, so that our discussion will be as efficient as the algorithms that we’ll be considering. Throughout this section we shall deal with variables, which are elements of any convenient set. Variables are often denoted by x_1, x_2, x_3, ..., as in (1); but any other symbols can also be used, like a, b, c, or even d74"′. We will in fact often use the numerals 1, 2, 3, ... to stand for variables; and in many cases we’ll find it convenient to write just j instead of xj, because it takes less time and less space if we don’t have to write so many x’s. Thus ‘2’ and ‘x_2’ will mean the same thing in many of the discussions below.

A literal is either a variable or the complement of a variable. In other words, if v is a variable, both v and v¯ are literals. If there are n possible variables in some problem, there are 2n possible literals. If l is the literal x¯2, which is also written 2¯, then the complement of l, denoted by l¯, is x_2, which is also written 2.

The variable that corresponds to a literal l is denoted by |l|; thus we have |v|=|v¯|=v for every variable v. Sometimes we write ±v for a literal that is either v or v¯. We might also denote such a literal by σv, where σ is ±1. The literal l is called positive if |l| = l; otherwise |l|=l¯, and l is said to be negative.

Two literals l and l′ are distinct if l ≠ l′. They are strictly distinct if |l| ≠ |l′|. A set of literals {l_1, ..., lk} is strictly distinct if |li| ≠ |lj| for 1 ≤ i < j ≤ k.

The satisfiability problem, like all good problems, can be understood in many equivalent ways, and we will find it convenient to switch from one viewpoint to another as we deal with different aspects of the problem. Example (1)isan AND of clauses, where every clause is an OR of literals; but we might as well regard every clause as simply a set of literals, and a formula as a set of clauses. With that simplification, and with ‘xj’ identical to ‘j’, Eq. (1) becomes

F={{1,2¯},{2,3},{1¯,3¯},{1¯,2¯,3}}.

And we needn’t bother to represent the clauses with braces and commas either; we can simply write out the literals of each clause. With that shorthand we’re able to perceive the real essence of (1) and (2):

F={12¯,23,1¯3¯,1¯2¯3},G=F∪{123¯}.(3)

Here F is a set of four clauses, and G is a set of five.

In this guise, the satisfiability problem is equivalent to a covering problem, analogous to the exact cover problems that we considered in Section 7.2.2.1: Let

Tn={{x1,x¯1},{x2,x¯2},...,{xn,x¯n}}={11¯,22¯,...,nn¯}.(4)

“Given a set F = {C_1, ..., Cm}, where each Ci is a clause and each clause consists of literals based on the variables {x_1, ..., xn}, find a set L of n literals that ‘covers’ F ∪ Tn, in the sense that every clause contains at least one element of L.” For example, the set F in (3) is covered by L={1¯,2¯,3}, and so is the set T_3; hence F is satisfiable. The set G is covered by {1,1¯,2} or {1,1¯,3} or ... or {2¯,3,3¯}, but not by any three literals that also cover T_3; so G is unsatisfiable.

Similarly, a family F of clauses is satisfiable if and only if it can be covered by a set L of strictly distinct literals.

If F′ is any formula obtained from F by complementing one or more variables, it’s clear that F′ is satisfiable if and only if F is satisfiable. For example, if we replace 1 by 1¯ and 2 by 2¯ in (3) we obtain

F′={1¯2,2¯3,13¯,123},G′=F′∪{1¯2¯3¯}.

In this case F′ is trivially satisfiable, because each of its clauses contains a positive literal: Every such formula is satisfied by simply letting L be the set of positive literals. Thus the satisfiability problem is the same as the problem of switching signs (or “polarities”) so that no all-negative clauses remain.

Another problem equivalent to satisfiability is obtained by going back to the Boolean interpretation in (1) and complementing both sides of the equation. By De Morgan’s laws 7.1.1–(11) and (12) we have

F¯(x1,x2,x3)=(x¯1∧x2)∨(x¯2∧x¯3)∨(x1∨x3)∨(x1∧x2∧x¯3);(5)

and F is unsatisfiable ⇔F=0⇔F¯=1=⇔F¯ is a tautology. Consequently F is satisfiable if and only if F¯ is not a tautology: The tautology problem and the satisfiability problem are essentially the same.*

* Strictly speaking, TAUT is coNP-complete, while SAT is NP-complete; see Section 7.9.

Since the satisfiability problem is so important, we simply call it SAT. And instances of the problem such as (1), in which there are no clauses of length greater than 3, are called 3SAT. In general, kSAT is the satisfiability problem restricted to instances where no clause has more than k literals.

Clauses of length 1 are called unit clauses, or unary clauses. Binary clauses, similarly, have length 2; then come ternary clauses, quaternary clauses, and so forth. Going the other way, the empty clause, or nullary clause, has length 0 and is denoted by ; it is always unsatisfiable. Short clauses are very important in algorithms for SAT, because they are easier to deal with than long clauses. But long clauses aren’t necessarily bad; they’re much easier to satisfy than the short ones.

A slight technicality arises when we consider clause length: The binary clause (x1∨x¯2) in (1) is equivalent to the ternary clause (x1∨x1⁢ ∨x¯2) as well as to (x1∨x¯2⁢ ∨x¯2) and to longer clauses such as (x1∨x1⁢ ∨x1∨x¯2); so we can regard it as a clause of any length ≥2. But when we think of clauses as sets of literals rather than ORs of literals, we usually rule out multisets such as 112¯ or 12¯2¯ that aren’t sets; in that sense a binary clause is not a special case of a ternary clause. On the other hand, every binary clause (x ∨ y) can be replaced by two ternary clauses, (x∨y∨z)∧(x∨y∨z¯), if z is another variable; and every k-ary clause is equivalent to the AND of two (k + 1)-ary clauses. Therefore we can assume, if we like, that kSAT deals only with clauses whose length is exactly k.

A clause is tautological (always satisfied) if it contains both v and v¯ for some variable v. Tautological clauses can be denoted by ℘ (see exercise 7.1.4–222). They never affect a satisfiability problem; so we usually assume that the clauses input to a SAT-solving algorithm consist of strictly distinct literals.

When we discussed the 3SAT problem briefly in Section 7.1.1, we took a look at formula 7.1.1–(32), “the shortest interesting formula in 3CNF.” In our new shorthand, it consists of the following eight unsatisfiable clauses:

R={123¯,234¯,341,41¯2,1¯2¯3,2¯3¯4,3¯4¯1¯,4¯12¯}.(6)

This set makes an excellent little test case, so we will refer to it frequently below. (The letter R reminds us that it is based on R. L. Rivest’s associative block design 6.5–(13).) The first seven clauses of R, namely

R′={123¯,234¯,341,41¯2,1¯2¯3,2¯3¯4,3¯4¯1¯},(7)

also make nice test data; they are satisfied only by choosing the complements of the literals in the omitted clause, namely {4,1¯,2}. More precisely, the literals 1¯, and 2 are necessary and sufficient to cover R′; we can also include either 3 or 3¯ in the solution. Notice that (6) is symmetric under the cyclic permutation 1→2→3→4→1¯→2¯→3¯→4¯→1 of literals; thus, omitting any clause of (6) gives a satisfiability problem equivalent to (7).

A simple example

SAT solvers are important because an enormous variety of problems can readily be formulated Booleanwise as ANDs of ORs. Let’s begin with a little puzzle that leads to an instructive family of example problems: Find a binary sequence x_1 ... x_8 that has no three equally spaced 0s and no three equally spaced 1s. For example, the sequence 01001011 almost works; but it doesn’t qualify, because x_2, x_5, and x_8 are equally spaced 1s.

If we try to solve this puzzle by backtracking manually through all 8-bit sequences in lexicographic order, we see that x_1x_2 = 00 forces x_3 = 1. Then x_1x_2x_3x_4x_5x_6x_7 = 0010011 leaves us with no choice for x_8. A minute or two of further hand calculation reveals that the puzzle has just six solutions, namely

00110011,01011010,01100110,10011001,10100101,11001100.(8)

Furthermore it’s easy to see that none of these solutions can be extended to a suitable binary sequence of length 9. We conclude that every binary sequence x_1 ... x_9 contains three equally spaced 0s or three equally spaced 1s.

Notice now that the condition x_2x_5x_8 ≠ 111 is the same as the Boolean clause (x¯2∨x¯5∨x¯8), namely 2¯5¯8¯. Similarly x_2x_5x_8 ≠ 000 is the same as 258. So we have just verified that the following 32 clauses are unsatisfiable:

123,234,...,789,135,246,...,579,147,258,369,1591¯2¯3¯,2¯3¯4¯,...,7¯8¯9¯,1¯3¯5¯,2¯4¯6¯,...,5¯7¯9¯,1¯4¯7¯,2¯5¯8¯,3¯6¯9¯,1¯5¯9¯.(9)

This result is a special case of a general fact that holds for any given positive integers j and k: If n is sufficiently large, every binary sequence x_1 ... xn contains either j equally spaced 0s or k equally spaced 1s (or both). The smallest such n is denoted by W (j, k) in honor of B. L. van der Waerden, who proved an even more general result (see exercise 2.3.4.3–6): If n is sufficiently large, and if k_0, ..., kb−1 are positive integers, every b-ary sequence x_1 ... xn contains ka equally spaced a’s for some digit a, 0 ≤ a < b. The least such n is W (k_0, ..., kb−1).

Let us accordingly define the following set of clauses when j, k, n > 0:

waerden(j,k;n)={(xi∨xi+d∨...∨xi+(j−1)d)|1≤i≤n−(j−1)d,d≥1}∪{x¯i∨x¯i+d∨...∨x¯i+(k−1)d|1≤i≤n−(k−1)d,d≥1}.(10)

The 32 clauses in (9) are waerden(3, 3; 9); and in general waerden(j, k; n) is an appealing instance of SAT, satisfiable if and only if n < W(j,k).

It’s obvious that W(1,k) = k and W(2,k)= 2k − [k even]; but when j and k exceed 2 the numbers W(j, k) are quite mysterious. We’ve seen that W (3, 3) = 9, and the following nontrivial values are currently known:

k=345678910111213141516171819W(3,k)=918223246587797114135160186218238279312349W(4,k)=18355573109146309??????????W(5,k)=2255178206260????????????W(6,k)=32732061132?????????????

V. Chvátal inaugurated the study of W(j, k) by computing the values for j+k ≤ 9 as well as W(3, 7) [Combinatorial Structures and Their Applications (1970), 31– 33]. Most of the large values in this table have been calculated by state-of-the-art SAT solvers [see M. Kouril and J. L. Paul, Experimental Math. 17 (2008), 53– 61; M. Kouril, Integers 12 (2012), A46:1–A46:13]. The table entries for j = 3 suggest that we might have W(3,k) < k^2 when k > 4, but that isn’t true: SAT solvers have also been used to establish the lower bounds
k=2021222324252627282930W(3,k)≥389416464516593656727770827868903

(which might in fact be the true values for this range of k); see T. Ahmed, O. Kullmann, and H. Snevily [Discrete Applied Math. 174 (2014), 27–51].

Notice that the literals in every clause of waerden (j, k; n) have the same sign: They’re either all positive or all negative. Does this “monotonic” property make the SAT problem any easier? Unfortunately, no: Exercise 10 proves that any set of clauses can be converted to an equivalent set of monotonic clauses.

Exact covering

The exact cover problems that we solved with “dancing links” in Section 7.2.2.1 can easily be reformulated as instances of SAT and handed off to SAT solvers. For example, let’s look again at Langford pairs, the task of placing two 1s, two 2s, ..., two n’s into 2n slots so that exactly k slots intervene between the two appearances of k, for each k. The corresponding exact cover problem when n = 3 has nine items and eight options (see 7.2.2.1–(17)):

′d1s1s3′,′d1s2s4′′d1s3s5′′d1s4s6′′d2s1s4′′d2s2s5′′d2s3s6′′d3s1s5′.(11)

The items are di for 1 ≤ i ≤ 3 and sj for 1 ≤ j ≤ 6; the option ‘di sj sk’ means that digit i is placed in slots j and k. Left-right symmetry allows us to omit the option ‘d_3 s_2 s_6’ from this specification.

We want to select options of (11) so that each item appears just once. Let the Boolean variable xj mean ‘select option j’, for 1 ≤ j ≤ 8; the problem is then to satisfy the nine constraints

S1(x1,x2,x3,x4)∧S1(x5,x6,x7)∧S1(x8)∧S1(x1,x5,x8)∧S1(x2,x6)∧S1(x1,x3,x7)∧S1(x2,x4,x5)∧S1(x3,x6,x8)∧S1(x4,x7),(12)

one for each item. (Here, as usual, S_1(y_1, ..., yp) denotes the symmetric function [y_1 + ... + yp = 1].) For example, we must have x_5 + x_6 + x_7 = 1, because item d_2 appears in options 5, 6, and 7 of (11).

One of the simplest ways to express the symmetric Boolean function S_1 as an AND of ORs is to use 1+(p2) clauses:

S1(y1,...,yp)=(y1∨...∨yp)∧∧(y¯j∨y¯k).(13)

“At least one of the y’s is true, but not two.” Then (12) becomes, in shorthand,

{1234,1¯2¯,1¯3,1¯4¯,2¯3¯,2¯4¯,3¯4¯,567,5¯6¯,5¯7¯,6¯7¯,8,158,1¯5¯,1¯8¯,5¯8¯,26,26¯,137,1¯3¯,1¯7¯,3¯7¯,⁢245,2¯4¯,2¯5¯,4¯5¯,368,3¯6¯,3¯8¯,6¯8¯,47,4¯7¯};(14)

we shall call these clauses langford (3). (Notice that only 30 of them are actually distinct, because 1¯3¯ and 2¯4¯ appear twice.) Exercise 13 defines langford (n); we know from exercise 7–1 that langford (n) is satisfiable ⇔ n mod 4 = 0 or 3.

The unary clause 8 in (14) tells us immediately that x_8 = 1. Then from the binary clauses 1¯8¯,5¯8¯,3¯8¯,6¯8¯ we have x_1 = x_5 = x_3 = x_6 = 0. The ternary clause 137 then implies x_7 = 1; finally x_4 = 0 (from 4¯7¯) and x_2 = 1 (from 1234). Options 8, 7, and 2 of (11) now give us the desired Langford pairing 312132.

Incidentally, the function S_1(y_1,y_2,y_3,y_4,y_5) can also be expressed as

(y1∨y2∨y3∨y4∨y5)∧(y¯1∨y¯2)∧(y¯1∨y¯3)∧(y¯1∨t¯)∧(y¯2∨y¯3)∧(y¯2∨t¯)∧(y¯3∨t¯)∧(t∨y¯4)∧(t∨y¯5)∧(y¯4∨y¯5),
where t is a new variable. In general, if p gets big, it’s possible to express S_1(y_1, ..., yp) with only 3p−5 clauses instead of (p2)+1, by using ⌊(p−3)/2⌋ new variables as explained in exercise 12. When this alternative encoding is used to represent Langford pairs of order n, we’ll call the resulting clauses langford (n).

Do SAT solvers do a better job with the clauses langford (n) or langford(n)? Stay tuned: We’ll find out later.

Coloring a graph

The classical problem of coloring a graph with at most d colors is another rich source of benchmark examples for SAT solvers. If the graph has n vertices V, we can introduce nd variables vj, for v ∊ V and 1 ≤ j ≤ d, signifying that v has color j; the resulting clauses are quite simple:

(v1∨v2∨...∨vd) for v∈V⁢  (“every vertex has at least one color”);(15)

(u¯j∨v¯j) for u—v,1≤j≤d⁢  (“adjacent vertices have different colors”).(16)

We could also add n(d2) additional so-called exclusion clauses

(v¯i∨v¯j) for v∈V,1≤i<j≤d⁢  (“every vertex has at most one color”);(17)

but they’re optional, because vertices with more than one color are harmless. Indeed, if we find a solution with v_1 = v_2 = 1, we’ll be extra happy, because it gives us two legal ways to color vertex v. (See exercise 14.)

Fig. 76. The McGregor graph of order 10. Each region of this “map” is identified by a two-digit hexadecimal code. Can you color the regions with four colors, never using the same color for two adjacent regions?

Martin Gardner astonished the world in 1975 when he reported [Scientific American 232, 4 (April 1975), 126–130] that a proper coloring of the planar map in Fig. 76 requires five distinct colors, thereby disproving the longstanding four-color conjecture. (In that same column he also cited several other “facts” supposedly discovered in 1974: (i) eπ163 is an integer; (ii) pawn-to-king-rook-4 (‘h4’) is a winning first move in chess; (iii) the theory of special relativity is fatally flawed; (iv) Leonardo da Vinci invented the flush toilet; and (v) Robert Ripoff devised a motor that is powered entirely by psychic energy. Thousands of readers failed to notice that they had been April Fooled!)

The map in Fig. 76 actually can be 4-colored; you are hereby challenged to discover a suitable way to do this, before turning to the answer of exercise 18. Indeed, the four-color conjecture became the Four Color Theorem in 1976, as mentioned in Section 7. Fortunately that result was still unknown in April of 1975; otherwise this interesting graph would probably never have appeared in print. McGregor’s graph has 110 vertices (regions) and 324 edges (adjacencies between regions); hence (15) and (16) yield 110 + 1296 = 1406 clauses on 440 variables, which a modern SAT solver can polish off quickly.

We can also go much further and solve problems that would be extremely difficult by hand. For example, we can add constraints to limit the number of regions that receive a particular color. Randal Bryant exploited this idea in 2010 to discover that there’s a four-coloring of Fig. 76 that uses one of the colors only 7 times (see exercise 17). His coloring is, in fact, unique, and it leads to an explicit way to 4-color the McGregor graphs of all orders n ≥ 3 (exercise 18).

Such additional constraints can be generated in many ways. We could, for instance, append (1108) clauses, one for every choice of 8 regions, specifying that those 8 regions aren’t all colored 1. But no, we’d better scratch that idea: (1108)=409,705,619,895. Even if we restricted ourselves to the 74,792,876,790 of 8 regions that are independent, we’d be dealing with far too many clauses.

An interesting SAT-oriented way to ensure that x_1 + ... + xn is at most r, which works well when n and r are rather large, was found by C. Sinz [LNCS 3709 (2005), 827–831]. His method introduces (n − r)r new variables sjk for 1 ≤ j ≤ n − r and 1 ≤ k ≤ r. If F is any satisfiability problem and if we add the (n − r − 1)r + (n − r)(r + 1) clauses

(s¯jk∨sj+1k),for 1 ≤j<n−r⁢   and 1≤k≤r,(18)

(x¯j+k∨sj−k∨sjk+1),for 1 ≤j≤n−r⁢   and 0 ≤k≤r,(19)

where sj−k is omitted when k = 0 and sjk+1 is omitted when k = r, then the new set of clauses is satisfiable if and only if F is satisfiable with x_1+...+ xn ≤ r. (See exercise 26.) With this scheme we can limit the number of red-colored regions of McGregor’s graph to at most 7 by appending 1538 clauses in 721 new variables.

Another way to achieve the same goal, which turns out to be even better, has been proposed by O. Bailleux and Y. Boufkhad [LNCS 2833 (2003), 108– 122]. Their method is a bit more difficult to describe, but still easy to implement: Consider a complete binary tree that has n−1 internal nodes numbered 1 through n − 1, and n leaves numbered n through 2n − 1; the children of node k, for 1 ≤ k < n, are nodes 2k and 2k+1 (see 2.3.4.5–(5)). We form new variables bjk for 1 < k < n and 1 ≤ j ≤ tk, where tk is the minimum of r and the number of leaves below node k. Then the following clauses, explained in exercise 27, do the job:

(b¯i2k∨b¯j2k+1∨bi+jk), for 0 ≤i≤t2k,0≤j≤t2k+1, 1≤i+j≤tk+1,1<k<n;(20)

(b¯i2∨b¯j3), for 0 ≤i≤t2,0≤j≤t3,i+j=r+1.(21)

In these formulas we let tk = 1 and b1k=xk−n+1 for n ≤ k < 2n; all literals b¯0k and br+1k are to be omitted. Applying (20) and (21) to McGregor’s graph, with n = 110 and r = 7, yields just 1216 new clauses in 399 new variables.

The same ideas apply when we want to ensure that x_1 +...+ xn is at least r, because of the identity S≥r(x1,...,xn)=S≤n−r(x¯1,...,x¯n). And exercise 30 considers the case of equality, when our goal is to make x_1 + ... + xn = r. We’ll discuss other encodings of such cardinality constraints below.

Factoring integers

Next on our agenda is a family of SAT instances with quite a different flavor. Given an (m + n)-bit binary integer z = (zm+n ...z_2z_1)2, do there exist integers x = (xm ...x_1)2 and y = (yn ...y_1)2 such that z = x × y? For example, if m = 2 and n = 3, we want to invert the binary multiplication
y3y2y1×x2x1¯a3a2a1b3b2b1¯c3c2c1¯z5z4z3z2⁢ z1(a3a2a1)2=(y3y2y1)2×x1(b3b2b1)2=(y3y2y1)2×x2z1=a1(c1z2)2=a2+b1(c2z3)2=a3+b2+c1(c3z4)2=b3+c2z5=c3(22)

when the z bits are given. This problem is satisfiable when z = 21 = (10101)2, in the sense that suitable binary values x_1, x_2, y_1, y_2, y_3, a_1, a_2, a_3, b_1, b_2, b_3, c_1, c_2, c_3 do satisfy these equations. But it’s unsatisfiable when z = 19 = (10011)2.

Arithmetical calculations like (22) are easily expressed in terms of clauses that can be fed to a SAT solver: We first specify the computation by constructing a Boolean chain, then we encode each step of the chain in terms of a few clauses. One such chain, if we identify a_1 with z_1 and c_3 with z_5, is
z1←x1∧y1,b1←x2∧y1,z2←a2⊕b1,s←a3⊕b2,z3←s⊕c1,z4←b3⊕c2,a2←x1∧y2,b2←x2∧y2,c1←a2∧b1,p←a3∧b2,q←s∧c1,z5←b3∧c2,a3←x1∧y3,b3←x2∧y3,c2←p∨q,(23)

using a “full adder” to compute c_2z_3 and “half adders” to compute c_1z_2 and c_3z_4 (see 7.1.2–(23) and (24)). And that chain is equivalent to the 49 clauses

(x1∨z¯1)∧(y1∨z¯1)∧(x¯1∨y¯1∨z1)∧...∧(b¯3∨c¯2∨z¯4)∧(b3∨z¯5)∧(c2∨z¯5)∧(b¯3∨c¯2∨z5)

obtained by expanding the elementary computations according to simple rules:

t←u∧v⁢ becomes (u∨t¯)∧(v∨t¯)∧(u¯∨v¯∨t);t←u∨v⁢ becomes (u¯∨t)∧(v¯∨t)∧(u∨v∨t¯);(24)t←u⊕v⁢ becomes (u¯∨v∨t)∧(u∨v¯∨t)∧(u∨v∨t¯)∧(u¯∨v¯∨t¯).

To complete the specification of this factoring problem when, say, z = (10101)2, we simply append the unary clauses (z5)∧(z¯4)(z3)∧(z¯2)∧(z1).

Logicians have known for a long time that computational steps can readily be expressed as conjunctions of clauses. Rules such as (24) are now called Tseytin encoding, after Gregory Tseytin (1966). Our representation of a small five-bit factorization problem in 49+5 clauses may not seem very efficient; but we will see shortly that m-bit by n-bit factorization corresponds to a satisfiability problem with fewer than 6mn variables, and fewer than 20mn clauses of length 3 or less.

Even if the system has hundreds or thousands of formulas,
it can be put into the conjunctive normal form “piece by piece”,
without any “multiplying out.”

— MARTIN DAVIS and HILARY PUTNAM (1958)

Suppose m ≤ n. The easiest way to set up Boolean chains for multiplication is probably to use a scheme that goes back to John Napier’s Rabdologiæ (Edinburgh, 1617), pages 137–143, as modernized by Luigi Dadda [Alta Frequenza 34 (1964), 349–356]: First we form all mn products xi ∧ yj, putting every such bit into bin [i + j], which is one of m + n “bins” that hold bits to be added for a particular power of 2 in the binary number system. The bins will contain respectively (0, 1, 2, ..., m, m, ..., m, ..., 2, 1) bits at this point, with n−m+1 occurrences of “m” in the middle. Now we look at bin [k] for k = 2, 3, .... If bin [k] contains a single bit b, we simply set zk−1 ← b. If it contains two bits {b,b′}, we use a half adder to compute zk−1←b⊕b′, c←b∧b′, and we put the carry bit c into bin [k + 1]. Otherwise bin [k] contains t ≥ 3 bits; we choose any three of them, say {b, b', b"}, and remove them from the bin. With a full adder we then compute r←b⊕b′⊕b" and c ← 〈bb′b″〉, so that b + b′ + b″ = r + 2c; and we put r into bin [k], c into bin [k+1]. This decreases t by 2, so eventually we will have computed zk−1. Exercise 41 quantifies the exact amount of calculation involved.

This method of encoding multiplication into clauses is quite flexible, since we’re allowed to choose any three bits from bin [k] whenever four or more bits are present. We could use a first-in-first-out strategy, always selecting bits from the “rear” and placing their sum at the “front”; or we could work last-in-first-out, essentially treating bin [k] as a stack instead of a queue. We could also select the bits randomly, to see if this makes our SAT solver any happier. Later in this section we’ll refer to the clauses that represent the factoring problem by calling them factor_fifo (m, n, z), factor_lifo (m, n, z), or factor_rand (m, n, z, s), respectively, where s is a seed for the random number generator used to generate them.

It’s somewhat mind-boggling to realize that numbers can be factored without using any number theory! No greatest common divisors, no applications of Fermat’s theorems, etc., are anywhere in sight. We’re providing no hints to the solver except for a bunch of Boolean formulas that operate almost blindly at the bit level. Yet factors are found.

Of course we can’t expect this method to compete with the sophisticated factorization algorithms of Section 4.5.4. But the problem of factoring does demonstrate the great versatility of clauses. And its clauses can be combined with other constraints that go well beyond any of the problems we’ve studied before.

Fault testing

Lots of things can go wrong when computer chips are manufactured in the “real world,” so engineers have long been interested in constructing test patterns to check the validity of a particular circuit. For example, suppose that all but one of the logical elements are functioning properly in some chip; the bad one, however, is stuck: Its output is constant, always the same regardless of the inputs that it is given. Such a failure is called a single-stuck-at fault.

Figure 77 illustrates a typical digital circuit in detail: It implements the 15 Boolean operations of (23) as a network that produces five output signals z_5z_4z_3z_2z_1 from the five inputs y_3y_2y_1x_2x_1. In addition to having 15 AND, OR, and XOR gates, each of which transforms two inputs into one output, it has 15 “fanout” gates (indicated by dots at junction points), each of which splits one input into two outputs. As a result it comprises 50 potentially distinct logical signals, one for each internal “wire.” Exercise 47 shows that a circuit with m outputs, n inputs, and g conventional 2-to-1 gates will have g + m − n fanout gates and 3g +2m − n wires. A circuit with w wires has 2w possible single-stuck-at faults, namely w faults in which the signal on a wire is stuck at 0 and w more on which it is stuck at 1.

Fig. 77. A circuit that corresponds to (23).

Table 1 shows 101 scenarios that are possible when the 50 wires of Fig. 77 are activated by one particular sequence of inputs, assuming that at most one stuck-at fault is present. The column headed OK shows the correct behavior of the Boolean chain (which nicely multiplies x = 3 by y = 6 and obtains z = 18). We can call these the “default” values, because, well, they have no faults. The other 100 columns show what happens if all but one of the 50 wires have error-free signals; the two columns under b21, for example, illustrate the results when the rightmost wire that fans out from gate b_2 is stuck at 0 or 1. Each row is obtained bitwise from previous rows or inputs, except that the boldface digits are forced. When a boldface value agrees with the default, its entire column is correct; otherwise errors might propagate. All values above the bold diagonal match the defaults.

Table 1
SINGLE-STUCK-AT FAULTS IN FIGURE 77 WHEN x_2x_1 = 11, y_3y_2y_1 = 110

If we want to test a chip that has n inputs and m outputs, we’re allowed to apply test patterns to the inputs and see what outputs are produced. Close inspection shows, for instance, that the pattern considered in Table 1 doesn’t detect an error when q is stuck at 1, even though q should be 0, because all five output bits z_5z_4z_3z_2z_1 are correct in spite of that error. In fact, the value of c_2 ← p ∨ q is unaffected by a bad q, because p = 1 in this example. Similarly, the fault “x12 stuck at 0” doesn’t propagate into z1←x12∧y11 because y11=0. Altogether 44 faults, not 50, are discovered by this particular test pattern.

All of the relevant repeatable faults, whether they’re single-stuck-at or wildly complicated, could obviously be discovered by testing all 2n possible patterns. But that’s out of the question unless n is quite small. Fortunately, testing isn’t hopeless, because satisfactory results are usually obtained in practice if we do have enough test patterns to detect all of the detectable single-stuck-at faults. Exercise 49 shows that just five patterns suffice to certify Fig. 77 by this criterion.

The detailed analysis in exercise 49 also shows, surprisingly, that one of the faults, namely “s^2 stuck at 1,” cannot be detected! Indeed, an erroneous s^2 can propagate to an erroneous q only if c12=1, and that forces x_1 = x_2 = y_1 = y_2 = 1; only two possibilities remain, and neither y_3 = 0 nor y_3 = 1 reveals the fault. Consequently we can simplify the circuit by removing gate q; the chain (23) becomes shorter, with “q ← s ∧ c_1, c_2 ← p ∨ q” replaced by “c_2 ← p ∨ c_1.”

Of course Fig. 77 is just a tiny little circuit, intended only to introduce the concept of stuck-at faults. Test patterns are needed for the much larger circuits that arise in real computers; and we will see that SAT solvers can help us to find them. Consider, for example, the generic multiplier circuit prod (m, n), which is part of the Stanford GraphBase. It multiplies an m-bit number x by an n-bit number y, producing an (m + n)-bit product z. Furthermore, it’s a so-called “parallel multiplier,” with delay time O(log(m + n)); thus it’s much more suited to hardware design than methods like the factor_fifo schemes that we considered above, because those circuits need Ω(m + n) time for carries to propagate.

Let’s try to find test patterns that will smoke out all of the single-stuck-at faults in prod (32, 32), which is a circuit of depth 33 that has 64 inputs, 64 outputs, 3660 AND gates, 1203 OR gates, 2145 XOR gates, and (therefore) 7008 fanout gates and 21,088 wires. How can we guard it against 42,176 different faults?

Before we construct clauses to facilitate that task, we should realize that most of the single-stuck-at faults are easily detected by choosing patterns at random, since faults usually cause big trouble and are hard to miss. Indeed, choosing x = # 3243F6A8 and y = # 885A308D more-or-less at random already eliminates 14,733 cases; and (x, y) = (#2B7E1516, #28AED2A6) eliminates 6,918 more. We might as well keep doing this, because bitwise operations such as those in Table 1 are fast. Experience with the smaller multiplier in Fig. 77 suggests that we get more effective tests if we bias the inputs, choosing each bit to be 1 with probability .9 instead of .5 (see exercise 49). A million such random inputs will then generate, say, 243 patterns that detect all but 140 of the faults.

Our remaining job, then, is essentially to find 140 needles in a haystack of size 2^64, after having picked 42,176 − 140 = 42,036 pieces of low-hanging fruit. And that’s where a SAT solver is useful. Consider, for example, the analogous but simpler problem of finding a test pattern for “q stuck at 0” in Fig. 77. We can use the 49 clauses F derived from (23) to represent the well-behaved circuit; and we can imagine corresponding clauses F′ that represent the faulty computation, using “primed” variables z1′,a2′,...,z5′. Thus F′ begins with (x1∨z¯1′)∧(y1∨z¯1′) and ends with (b¯3′∨c¯2′∨z5′); it’s like F except that the clauses representing q′←s′∧c1′ in (23) are changed to simply q¯′ (meaning that q′ is stuck at 0). Then the clauses of F and F′, together with a few more clauses to state that z1≠z1′ or ... or z5≠z5′, will be satisfiable only by variables for which (y_3y_2y_1)2 × (x_2x_1)2 is a suitable test pattern for the given fault.

This construction of F′ can obviously be simplified, because z1′ is identical to z_1; any signal that differs from the correct value must be located “downstream” from the one-and-only fault. Let’s say that a wire is tarnished if it is the faulty wire or if at least one of its input wires is tarnished. We introduce new variables g′ only for wires g that are tarnished. Thus, in our example, the only clauses F′ that are needed to extend F to a faulty companion circuit are q¯′ and the clauses that correspond to c2′←p∨q′,z4′←b3⊕c2′,z5′←b3∧c2′..

Moreover, any fault that is revealed by a test pattern must have an active path of wires, leading from the fault to an output; all wires on this path must carry a faulty signal. Therefore Tracy Larrabee [IEEE Trans. CAD-11 (1992), 4–15] decided to introduce additional “sharped” variables g# for each tarnished wire, meaning that g lies on the active path. The two clauses

(g¯#∨g∨g′)∧(g¯#∨g¯∨g¯′)(25)

ensure that g ≠ g′ whenever g is part of that path. Furthermore we have (v¯#∨g#) whenever g is an AND, OR, or XOR gate with tarnished input v. Fanout gates are slightly tricky in this regard: When wires g^1 and g^2 fan out from a tarnished wire g, we need variables g1# and g2# as well as g#; and we introduce the clause

(g¯#∨g1#∨g2#)(26)

to specify that the active path takes at least one of the two branches.

According to these rules, our example acquires the new variables q#,c2#,c21#,c22#,z4#,z5#, and the new clauses

(q¯#∨q∨q′)∧(q¯#∨q¯∨q¯′)∧(q¯#∨c2#)∧(c¯2#∨c2∨c2′)∧(c¯2#∨c¯2∨c¯2′)∧(c¯2#∨c21#∨c22#)∧(c¯21#∨z4#)∧(z¯4#∨z4∨z4′)∧(z¯4#∨z¯4∨z¯4′)∧(c¯22#∨z5#)∧(z¯5#∨z5∨z5′)∧(z¯5#∨z¯5∨z¯5′).

The active path begins at q, so we assert the unit clause (q#); it ends at a tarnished output, so we also assert (z4#∨z5#). The resulting set of clauses will find a test pattern for this fault if and only if the fault is detectable. Larrabee found that such active-path variables provide important clues to a SAT solver and significantly speed up the solution process.

Returning to the large circuit prod (32, 32), one of the 140 hard-to-test faults is “W2126 stuck at 1,” where W2126 denotes the 26th extra wire that fans out from the OR gate called W_21 in §75 of the Stanford GraphBase program GB_GATES; W2126 is an input to gate b4040←d4019∧W2126 in §80 of that program. Test patterns for that fault can be characterized by a set of 23,194 clauses in 7,082 variables (of which only 4 variables are “primed” and 4 are “sharped”). Fortunately the solution (x, y) = (#7F13FEDD, #5FE57FFE) was found rather quickly in the author’s experiments; and this pattern also killed off 13 of the other cases, so the score was now “14 down and 126 to go”!

The next fault sought was “A536,2 stuck at 1,” where A536,2 is the second extra wire to fan out from the AND gate A536 in §72 of GB GATES (an input to R1136←A536,2∧R135,2). This fault corresponds to 26,131 clauses on 8,342 variables; but the SAT solver took a quick look at those clauses and decided almost instantly that they are unsatisfiable. Therefore the fault is undetectable, and the circuit prod (32, 32) can be simplified by setting R1136←R135,2. A closer look showed, in fact, that clauses corresponding to the Boolean equations
x=y∧z,y=v∧w,z=t∧u,u=v⊕w
were present (where t=R1344,u=A5845,v=R444,w=A1445,x=R2346,y=R1345,z=R1945); these clauses force x = 0. Therefore it was not surprising to find that the list of unresolved faults also included R2346,R2346,1, and R2346,2 stuck at 0. Altogether 26 of the 140 faults undetected by random inputs turned out to be absolutely undetectable; and only one of these, namely “Q2646 stuck at 0,” required a nontrivial proof of undetectability.

Some of the 126−26 = 100 faults remaining on the to-do list turned out to be significant challenges for the SAT solver. While waiting, the author therefore had time to take a look at a few of the previously found solutions, and noticed that those patterns themselves were forming a pattern! Sure enough, the extreme portions of this large and complicated circuit actually have a fairly simple structure, stuck-at-fault-wise. Hence number theory came to the rescue: The factorization #87FBC059 × F0F87817 = 2^63 − 1 solved many of the toughest challenges, some of which occur with probability less than 2−34 when 32-bit numbers are multiplied; and the “Aurifeuillian” factorization (2^31 − 2^16 +1)(2^31 +2^16 +1) = 2^62 + 1, which the author had known for more than forty years (see Eq. 4.5.4– (15)), polished off most of the others.

The bottom line (see exercise 51) is that all 42,150 of the detectable single-stuck-at faults of the parallel multiplication circuit prod (32, 32) can actually be detected with at most 196 well-chosen test patterns.

Learning a Boolean function

Sometimes we’re given a “black box” that evaluates a Boolean function f(x_1, ..., xN ). We have no way to open the box, but we suspect that the function is actually quite simple. By plugging in various values for x = x_1 ...xN , we can observe the box’s behavior and possibly learn the hidden rule that lies inside. For example, a secret function of N = 20 Boolean variables might take on the values shown in Table 2, which lists 16 cases where f(x) = 1 and 16 cases where f(x)= 0.

Suppose we assume that the function has a DNF (disjunctive normal form) with only a few terms. We’ll see in a moment that it’s easy to express such an assumption as a satisfiability problem. And when the author constructed clauses corresponding to Table 2 and presented them to a SAT solver, he did in fact learn almost immediately that a very simple formula is consistent with all of the data:

f(x1,...,x20)=x¯2x¯3x¯10∨x¯6x¯10x¯12∨x8x¯13x¯15∨x¯8x10x¯12.(27)

Table 2
VALUES TAKEN ON BY AN UNKNOWN FUNCTION

Cases where f(x) = 1

Cases where f(x) = 0

x_1 x_2 x_3 x_4 x_5 x_6 x_7 x_8 x_9     . . .     x_20

x_1 x_2 x_3 x_4 x_5 x_6 x_7 x_8 x_9     . . .     x_20

1 1 0 0 1 0 0 1 0 0 0 0 1 1 1 1 1 1 0 1

1 0 1 0 1 0 1 0 0 0 1 0 0 0 1 0 0 0 0 1

0 1 1 0 1 0 0 0 1 1 0 0 0 0 1 0 0 0 1 1

0 1 0 0 1 1 0 0 0 1 0 0 1 1 0 0 0 1 1 0

0 1 1 0 0 0 1 0 1 0 0 0 1 0 1 1 1 0 0 0

0 0 0 0 1 1 0 1 1 1 0 0 0 0 0 1 1 1 0 0

1 1 0 1 0 0 0 1 0 0 1 0 1 0 0 1 0 0 0 0

0 0 1 0 0 1 0 0 1 1 1 0 0 0 0 0 1 0 0 0

1 0 0 0 1 0 1 0 0 1 1 0 0 1 1 1 1 1 0 0

1 1 0 0 0 1 1 1 0 1 0 0 0 0 0 0 0 0 1 0

0 0 0 0 1 0 1 1 1 0 1 1 1 1 1 0 1 0 1 0

0 1 1 0 0 0 1 1 1 0 1 1 0 0 0 1 0 0 1 1

1 0 0 1 1 0 1 1 0 0 1 0 0 0 1 0 0 1 0 1

0 0 0 1 0 1 0 0 1 0 1 0 0 0 0 0 1 0 0 0

0 1 1 1 1 0 0 1 1 0 0 0 1 1 1 0 0 0 1 1

0 1 0 0 0 0 0 0 0 1 0 0 1 1 0 1 1 1 0 1

1 0 1 0 1 1 0 1 1 1 1 1 1 0 0 0 0 1 0 1

0 1 0 0 0 1 0 1 1 0 0 0 1 0 1 0 0 0 1 0

1 0 1 1 1 0 1 1 0 1 0 0 1 0 1 0 1 0 0 1

1 0 1 0 1 0 1 0 1 1 1 1 1 1 0 1 1 1 0 0

0 1 0 1 0 1 1 0 0 0 1 0 0 0 0 0 0 0 1 0

0 1 1 1 0 0 1 1 1 1 0 1 0 0 1 1 1 1 0 0

1 1 1 1 0 0 0 1 1 1 0 1 1 0 0 0 1 0 1 1

1 0 0 1 1 1 0 0 0 1 0 1 1 0 0 0 0 0 1 1

1 1 0 0 1 1 1 0 0 0 1 0 1 1 0 1 0 0 1 1

0 1 1 0 1 0 0 1 0 1 0 1 1 0 1 0 1 0 0 1

1 1 1 0 0 0 0 1 0 0 1 1 0 1 1 0 0 1 0 0

0 0 0 1 0 0 0 1 0 1 0 0 0 1 1 0 0 1 0 0

0 0 1 1 0 0 1 1 1 1 1 1 1 0 1 1 1 1 0 0

1 1 0 0 1 0 0 1 0 0 1 1 1 0 0 1 1 1 0 1

1 1 0 0 1 1 1 0 0 0 1 0 0 1 0 0 1 0 0 1

1 0 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 0 0 1

This formula was discovered by constructing clauses in 2MN variables pi,j and qi,j for 1 ≤ i ≤ M and 1 ≤ j ≤ N, where M is the maximum number of terms allowed in the DNF (here M = 4) and where
pi,j=[term i⁢  contains xj],qi,j=[term i⁢  contains x¯j].(28)

If the function is constrained to equal 1 at P specified points, we also use auxiliary variables zi,k for 1 ≤ i ≤ M and 1 ≤ k ≤ P, one for each term at every such point.

Table 2 says that f(1, 1, 0, 0, ..., 1) = 1, and we can capture this specification by constructing the clause

(z1,1∨z2,1∨...∨zM,1)(29)

together with the clauses

(z¯i,1∨q¯i,1)∧(z¯i,1∨q¯i,2)∧(z¯i,1∨p¯i,3)∧(z¯i,1∨p¯i,4)∧...∧(z¯i,1∨q¯i,20)(30)

for 1 ≤ i ≤ M. Translation: (29) says that at least one of the terms in the DNF must evaluate to true; and (30) says that, if term i is true at the point 1100 ... 1, it cannot contain x¯1 or x¯2 or x_3 or x_4 or ... or x¯20.

Table 2 also tells us that f(1, 0, 1, 0, ..., 1) = 0. This specification corresponds to the clauses

(qi,1∨pi,2∨qi,3∨pi,4∨...∨qi,20)(31)

for 1 ≤ i ≤ M. (Each term of the DNF must be zero at the given point; thus either x¯1 or x_2 or x¯3 or x_4 or ... or x¯20 must be present for each value of i.)

In general, every case where f(x) = 1 yields one clause like (29) of length M, plus MN clauses like (30) of length 2. Every case where f(x) = 0 yields M clauses like (31) of length N. We use qi,j when xj = 1 at the point in question, and pi,j when xj = 0, for both (30) and (31). This construction is due to A. P. Kamath, N. K. Karmarkar, K. G. Ramakrishnan, and M. G. C. Resende [Mathematical Programming 57 (1992), 215–238], who presented numerous examples. From Table 2, with M = 4, N = 20, and P = 16, it generates 1360 clauses of total length 3904 in 224 variables; a SAT solver then finds a solution with p_1,1 = q_1,1 = p_1,2 = 0, q_1,2 = 1, ..., leading to (27).

The simplicity of (27) makes it plausible that the SAT solver has indeed psyched out the true nature of the hidden function f(x). The chance of agreeing with the correct value 32 times out of 32 is only 1 in 2^32, so we seem to have overwhelming evidence in favor of that equation.

But no: Such reasoning is fallacious. The numbers in Table 2 actually arose in a completely different way, and Eq. (27) has essentially no credibility as a predictor of f(x) for any other values of x! (See exercise 53.) The fallacy comes from the fact that short-DNF Boolean functions of 20 variables are not at all rare; there are many more than 2^32 of them.

D. Morgenstern has found a much simpler formula that also matches Table 2:

f(x1,...,x20)=x¯4x10x¯12∨x¯6x¯10x¯12∨x9x¯10x11.

But it’s actually further than (27) from the “true”f that’s revealed in exercise 53.

On the other hand, when we do know that the hidden function f(x) has a DNF with at most M terms (although we know nothing else about it), the clauses (29)–(31) give us a nice way to discover those terms, provided that we also have a sufficiently large and unbiased “training set” of observed values.

For example, let’s assume that (27) actually is the function in the box. If we examine f(x) at 32 random points x, we don’t have enough data to make any deductions. But 100 random training points will almost always home in on the correct solution (27). This calculation typically involves 3942 clauses in 344 variables; yet it goes quickly, needing only about 100 million accesses to memory.

One of the author’s experiments with a 100-element training set yielded
f^(x1,...,x20)=x¯2x¯3x¯10∨x3x¯6x¯10x¯12∨x8x¯13x15∨x¯8x10x¯12,(32)

which is close to the truth but not quite exact. (Exercise 59 proves that f^(x) is equal to f(x) more than 97% of the time.) Further study of this example showed that another nine training points were enough to deduce f(x) uniquely, thus obtaining 100% confidence (see exercise 61).

Bounded model checking

Some of the most important applications of SAT solvers in practice are related to the verification of hardware or software, because designers generally want some kind of assurance that particular implementations correctly meet their specifications.

A typical design can usually be modeled as a transition relation between Boolean vectors X = x_1 ...xn that represent the possible states of a system. We write X→X′ if state X at time t can be followed by state X′ at time t + 1. The task in general is to study sequences of state transitions

X0→X1→X2→...→Xr,(33)

and to decide whether or not there are sequences that have special properties. For example, we hope that there’s no such sequence for which X_0 is an “initial state” and Xr is an “error state”; otherwise there’d be a bug in the design.

Fig. 78. Conway’s rule (35) defines these three successive transitions.

Questions like this are readily expressed as satisfiability problems: Each state Xt is a vector of Boolean variables xt_1 ...xtn, and each transition relation can be represented by a set of m clauses T (Xt,Xt+1) that must be satisfied. These clauses T(X, X′) involve 2n variables {x1,...,xn,x1′,...,xn′}, together with q auxiliary variables {y_1, ..., yq} that might be needed to express Boolean formulas in clause form as we did with the Tseytin encodings in (24). Then the existence of sequence (33) is equivalent to the satisfiability of mr clauses

T(X0,X1)∧T(X1,X2)∧T(Xr−1,Xr)(34)

in the n(r+1)+ qr variables {xtj | 0 ≤ t ≤ r, 1 ≤ j ≤n} ∪ {ytk | 0≤t<r, 1≤k ≤q}. We’ve essentially “unrolled” the sequence (33) into r copies of the transition relation, using variables xtj for state Xt and ytk for the auxiliary quantities in T (Xt,Xt+1). Additional clauses can now be added to specify constraints on the initial state X_0 and/or the final state Xr, as well as any other conditions that we want to impose on the sequence.

This general setup is called “bounded model checking,” because we’re using it to check properties of a model (a transition relation), and because we’re considering only sequences that have a bounded number of transitions, r.

John Conway’s fascinating Game of Life provides a particularly instructive set of examples that illustrate basic principles of bounded model checking. The states X of this game are two-dimensional bitmaps, corresponding to arrays of square cells that are either alive (1) or dead (0). Every bitmap X has a unique successor X′, determined by the action of a simple 3 × 3 cellular automaton: Suppose cell x has the eight neighbors {xNW, xN, xNE, xW, xE, xSW, xS, xSE}, and let ν = xNW + xN + xNE + xW + xE + xSW + xS + xSE be the number of neighbors that are alive at time t. Then x is alive at time t + 1 if and only if either (a) ν = 3, or (b) ν = 2 and x is alive at time t. Equivalently, the transition rule
x′=[2<xNW+xN+xNE+xW+12x+xE+xSW+xS+xSE<4](35)

holds at every cell x. (See, for example, Fig. 78, where the live cells are black.)

Conway called Life a “no-player game,” because it involves no strategy: Once an initial state X_0 has been set up, all subsequent states X_1, X_2, ... are completely determined. Yet, in spite of the simple rules, he also proved that Life is inherently complicated and unpredictable, indeed beyond human comprehension, in the sense that it is universal: Every finite, discrete, deterministic system, however complex, can be simulated faithfully by some finite initial state X_0 of Life. [See Berlekamp, Conway, and Guy, Winning Ways (2004), Chapter 25.]

In exercises 7.1.4–160 through 162, we’ve already seen some of the amazing Life histories that are possible, using BDD methods. And many further aspects of Life can be explored with SAT methods, because SAT solvers can often deal with many more variables. For example, Fig. 78 was discovered by using 7×15 = 105 variables for each state X_0, X_1, X_2, X_3. The values of X_3 were obviously predetermined; but the other 105 × 3 = 315 variables had to be computed, and BDDs can’t handle that many. Moreover, additional variables were introduced to ensure that the initial state X_0 would have as few live cells as possible.

Here’s the story behind Fig. 78, in more detail: Since Life is two-dimensional, we use variables xij instead of xj to indicate the states of individual cells, and xtij instead of xtj to indicate the states of cells at time t. We generally assume that xtij = 0 for all cells outside of a given finite region, although the transition rule (35) can allow cells that are arbitrarily far away to become alive as Life goes on. In Fig. 78 the region was specified to be a 7 × 15 rectangle at each unit of time. Furthermore, configurations with three consecutive live cells on a boundary edge were forbidden, so that cells “outside the box” wouldn’t be activated.

The transitions T (Xt,Xt+1) can be encoded without introducing additional variables, but only if we introduce 190 rather long clauses for each cell not on the boundary. There’s a better way, based on the binary tree approach underlying (20) and (21) above, which requires only about 63 clauses of size ≤ 3, together with about 14 auxiliary variables per cell. This approach (see exercise 65) takes advantage of the fact that many intermediate calculations can be shared. For example, cells x and xW have four neighbors {xNW,xN,xSW,xS} in common; so we need to compute xNW + xN + xSW + xS only once, not twice.

The clauses that correspond to a four-step sequence X_0 → X_1 → X_2 → X_3 → X_4 leading to X_4 = turn out to be unsatisfiable without going outside of the 7 × 15 frame. (Only 10 gigamems of calculation were needed to establish this fact, using Algorithm C below, even though roughly 34000 clauses in 9000 variables needed to be examined!) So the next step in the preparation of Fig. 78 was to try X_3 = ; and this trial succeeded. Additional clauses, which permitted X_0 to have at most 39 live cells, led to the solution shown, at a cost of about 17 gigamems; and that solution is optimum, because a further run (costing 12 gigamems) proved that there’s no solution with at most 38.

Let’s look for a moment at some of the patterns that can occur on a chessboard, an 8 × 8 grid. Human beings will never be able to contemplate more than a tiny fraction of the 2^64 states that are possible; so we can be fairly sure that “Lifenthusiasts” haven’t already explored every tantalizing configuration that exists, even on such a small playing field.

One nice way to look for a sequence of interesting Life transitions is to assert that no cell stays alive more than four steps in a row. Let us therefore say that a mobile Life path is a sequence of transitions X_0 → X_1 → ... → Xr with the additional property that we have

(x¯tij∨x¯(t+1)ij∨x¯(t+2)ij∨x¯(t+3)ij∨x¯(t+4)ij),for 0≤t≤r−4.(36)

To avoid trivial solutions we also insist that Xr is not entirely dead. For example, if we impose rule (36) on a chessboard, with xtij permitted to be alive only if 1 ≤ i, j ≤ 8, and with the further condition that at most five cells are alive in each generation, a SAT solver can quickly discover interesting mobile paths such as

which last quite awhile before leaving the board. And indeed, the five-celled object that moves so gracefully in this path is R. K. Guy’s famous glider (1970), which is surely the most interesting small creature in Life’s universe. The glider moves diagonally, recreating a shifted copy of itself after every four steps.

Interesting mobile paths appear also if we restrict the population at each time to {6, 7, 8, 9, 10} instead of {1, 2, 3, 4, 5}. For example, here are some of the first such paths that the author’s solver came up with, having length r = 8:

These paths illustrate the fact that symmetry can be gained, but never lost, as Life evolves deterministically. Marvelous designs are spawned in the process. In each of these sequences the next bitmap, X_9, would break our ground rules: The population immediately after X_8 grows to 12 in the first and last examples, but shrinks to 5 in the second-from-last; and the path becomes immobile in the other two. Indeed, we have X_5 = X_7 in the second example, hence X_6 = X_8 and X_7 = X_9, etc. Such a repeating pattern is called an oscillator of period 2. The third example ends with an oscillator of period 1, known as a “still life.”

What are the ultimate destinations of these paths? The first one becomes still, with X_69 = X_70; and the fourth becomes very still, with X_12 = 0! The fifth is the most fascinating of the group, because it continues to produce ever more elaborate valentine shapes, then proceeds to dance and sparkle, until finally beginning to twinkle with period 2 starting at time 177. Thus its members X_2 through X_7 qualify as “Methuselahs,” defined by Martin Gardner as “Life patterns of population less than 10 that do not become stable within 50 generations.” (A repetitive pattern, like the glider or an oscillator, is called stable.)

SAT solvers are basically useless for the study of Methuselahs, because the state space becomes too large. But they are quite helpful when we want to illuminate many other aspects of Life, and exercises 66–85 discuss some notable instances. We will consider one more instructive example before moving on, namely an application to “eaters.” Consider a Life path of the form

where the gray cells form a still life and the cells of X_1, X_2, X_3 are unknown. Thus X_4 = X_5 and X_0 = X_5 + glider. Furthermore we require that the still life X_5 does not interact with the glider’s parent, ; see exercise 77. The idea is that a glider will be gobbled up if it happens to glide into this particular still life, and the still life will rapidly reconstitute itself as if nothing had happened.

Algorithm C almost instantaneously (well, after about 100 megamems) finds

the four-step eater that was first observed in action by R. W. Gosper in 1971.

Applications to mutual exclusion

Let’s look now at how bounded model checking can help us to prove that algorithms are correct. (Or incorrect.) Some of the most challenging issues of verification arise when we consider parallel processes that need to synchronize their concurrent behavior. To simplify our discussion it will be convenient to tell a little story about Alice and Bob.

Alice and Bob are casual friends who share an apartment. One of their joint rooms is special: When they’re in that critical room, which has two doors, they don’t want the other person to be present. Furthermore, being busy people, they don’t want to interrupt each other needlessly. So they agree to control access to the room by using an indicator light, which can be switched on or off.

The first protocol they tried can be characterized by symmetrical algorithms:

A0. Maybe go to A1.B0. Maybe go to B1.A1. If l go to A1, else to A2.B1. If l go to B1, else to B2.A2. Set l ← 1, go to A3.B2. Set l ← 1, go to B3.(40)A3. Critical, go to A4.B3. Critical, go to B4.A4. Set l ← 0, go to A0.B4. Set l→0, go to B0.

At any instant of time, Alice is in one of five states, {A0, A1, A2, A3, A4}, and the rules of her program show how that state might change. In state A0 she isn’t interested in the critical room; but she goes to A1 when she does wish to use it. She reaches that objective in state A3. Similar remarks apply to Bob. When the indicator light is on (l = 1), they wait until the other person has exited the room and switched the light back off (l = 0).

Alice and Bob don’t necessarily operate at the same speed. But they’re allowed to dawdle only when in the “maybe” state A0 or B0. More precisely, we model the situation by converting every relevant scenario into a discrete sequence of state transitions. At every time t = 0, 1, 2, ..., either Alice or Bob (but not both) will perform the command associated with their current state, thereby perhaps changing to a different state at time t + 1. This choice is nondeterministic.

Only four kinds of primitive commands are permitted in the procedures we shall study, all of which are illustrated in (40): (1) “Maybe go to s”; (2) “Critical, go to s”; (3) “Set v ← b, go to s”; and (4) “If v go to s_1, else to s_0”. Here s denotes a state name, v denotes a shared Boolean variable, and b is 0 or 1.

Unfortunately, Alice and Bob soon learned that protocol (40) is unreliable: One day she went from A1 to A2 and he went from B1 to B2, before either of them had switched the indicator on. Embarrassment (A3 and B3) followed.

They could have discovered this problem in advance, if they’d converted the state transitions of (40) into clauses for bounded model checking, as in (33), then applied a SAT solver. In this case the vector Xt that corresponds to time t consists of Boolean variables that encode each of their current states, as well as the current value of l. We can, for example, have eleven variables A0t, A1t, A2t, A3t, A4t, B0t, B1t, B2t, B3t, B4t, lt, together with ten binary exclusion clauses (A0t¯⁢∨ A1t¯),(A0t¯⁢∨ A2t¯),...,(A3t¯⁢∨ A4t¯) to ensure that Alice is in at most one state, and with ten similar clauses for Bob. There’s also a variable @t, which is true or false depending on whether Alice or Bob executes their program step at time t. (We say that Alice was “bumped” if @t = 1, and Bob was bumped if @t = 0.)

If we start with the initial state X_0 defined by unit clauses

A00∧A10¯∧A20¯∧A30¯∧A40¯∧B00∧B10¯∧B20¯∧B30¯∧B40¯∧l¯0,(41)

the following clauses for 0 ≤ t < r (discussed in exercise 87) will emulate the first r steps of every legitimate scenario defined by (40):

(@t∨A0¯t∨A0t+1)(@t¯∨A0t¯∨A0t+1∨A1t+1)(@t∨B0t¯∨B0t+1∨B1t+1)(@t∨A1¯t∨A1t+1)(@t¯∨A1t¯∨lt¯∨A1t+1)(@t∨B1t¯∨l¯t∨B1t+1)(@t∨A2¯t∨A2t+1)(@t¯∨A1t¯∨lt∨A2t+1)(@t∨B1t¯∨lt∨B2t+1)(@t∨A3¯t∨A3t+1)(@t¯∨A2t¯∨A3t+1)(@t∨B2t¯∨B3t+1)(@t∨A4¯t∨A4t+1)(@t¯∨A2t¯∨lt+1)(@t∨B2t¯∨lt+1)(42)(@t¯∨B0¯t∨B0t+1)(@t¯∨A3t¯∨A4t+1)(@t∨B3t¯∨B4t+1)(@t¯∨B1¯t∨B1t+1)(@t¯∨A4t¯∨A0t+1)(@t∨B4t¯∨B0t+1)(@t¯∨B2¯t∨B2t+1)(@t¯∨A4t¯∨lt+1)(@t∨B4t¯∨l¯t+1)(@t¯∨B3¯t∨B3t+1)(@t¯∨lt∨A2t∨A4t∨l¯t+1)(@t∨lt∨B2t∨B4t∨l¯t+1)(@t¯∨B4¯t∨B4t+1)(@t¯∨l¯t∨A2t∨A4t∨lt+1)(@t∨l¯t∨B2t∨B4t∨lt+1)

If we now add the unit clauses (A3r) and (B3r), the resulting set of 13 + 50r clauses in 11+12r variables is readily satisfiable when r = 6, thereby proving that the critical room might indeed be jointly occupied. (Incidentally, standard terminology for mutual exclusion protocols would say that “two threads concurrently execute a critical section”; but we shall continue with our roommate metaphor.)

Back at the drawing board, one idea is to modify (40) by letting Alice use the room only when l = 1, but letting Bob in when l = 0:

A0. Maybe go to A1.A1. If l⁢  go to A2, else to A1.A2. Critical, go to A3.A3. Set l←0, go to A0.B0. Maybe go to B1.B1. If l⁢   go to B1, else to B2.B2. Critical, go to B3.B3. Set l←1, go to B0.(43)

Computer tests with r = 100 show that the corresponding clauses are unsatisfiable; thus mutual exclusion is apparently guaranteed by (43).

But (43) is a nonstarter, because it imposes an intolerable cost: Alice can’t use the room k times until Bob has already done so! Scrap that.

How about installing another light, so that each person controls one of them?

A0. Maybe go to A1.A1. If b⁢  go to A1, else to A2.A2. Set a←1, go to A3.A3.  Critical,  go to A4.A4. Set a←0, go to A0. B0. Maybe go to B1.B1. If a⁢   go to B1, else to B2.B2. Set b←1, go to B3.B3. Critical, go to B4.B4.  Set b←0, go to B0.(44)

No; this suffers from the same defect as (40). But maybe we can cleverly switch the order of steps 1 and 2:

A0. Maybe go to A1.A1. Set a←1,go to A2.A2. If b⁢  go to A2, else to A3.A3.  Critical,  go to A4.A4. Set a←0, go to A0. B0. Maybe go to B1.B1. Set b←1, go to B2.B2. If a⁢ go to B2, else to B3.B3. Critical, go to B4.B4.  Set b←0, go to B0.(45)

Yes! Exercise 95 proves easily that this protocol does achieve mutual exclusion.

Alas, however, a new problem now arises, namely the problem known as “deadlock” or “livelock.” Alice and Bob can get into states A2 and B2, after which they’re stuck — each waiting for the other to go critical.

In such cases they could agree to “reboot” somehow. But that would be a cop-out; they really seek a better solution. And they aren’t alone: Many people have struggled with this surprisingly delicate problem over the years, and several solutions (both good and bad) appear in the exercises below. Edsger Dijkstra, in some pioneering lecture notes entitled Cooperating Sequential Processes [Technological University Eindhoven (September 1965), §2.1], thought of an instructive way to improve on (45):

A0. Maybe go to A1.A1. Set a←1, go to A2.A2. If b⁢  go to A3, else to A4.A3. Set a←0, go to A1.A4. Critical, go to A5.A5. Set a←0, go to A0.B0. Maybe go to B1.B1. Set b←1, go to B2.B2. If a⁢  go to B3, else to B4.B3. Set b←0, go to B1.B4. Critical, go to B5.B5. Set b←0, go to B0.(46)

But he realized that this too is unsatisfactory, because it permits scenarios in which Alice, say, might wait forever while Bob repeatedly uses the critical room. (Indeed, if Alice and Bob are in states A1 and B2, she might go to A2, A3, then A1, thereby letting him run to B4, B5, B0, B1, and B2; they’re back where they started, yet she’s made no progress.)

The existence of this problem, called starvation, can also be detected via bounded model checking. The basic idea (see exercise 91) is that starvation occurs if and only if there is a loop of transitions

X0→X1→...→Xp→Xp+1→...→Xr=Xp(47)

such that (i) Alice and Bob each are bumped at least once during the loop; and (ii) at least one of them is never in a “maybe” or “critical” state during the loop. And those conditions are easily encoded into clauses, because we can identify the variables for time r with the variables for time p, and we can append the clauses

(@p¯∨@p+1¯∨...∨@r−1¯)∧(@p∨@p+1∨...∨@r−1)(48)

to guarantee (i). Condition (ii) is simply a matter of appending unit clauses; for example, to test whether Alice can be starved by (46), the relevant clauses are A0p¯∧A0p+1¯∧...∧A0r−1¯∧A4p¯∧A4p+1¯∧...∧A4r−1¯.

The deficiencies of (43), (45), and (46) can all be viewed as instances of starvation, because (47) and (48) are satisfiable (see exercise 90). Thus we can use bounded model checking to find counterexamples to any unsatisfactory protocol for mutual exclusion, either by exhibiting a scenario in which Alice and Bob are both in the critical room or by exhibiting a feasible starvation cycle (47).

Of course we’d like to go the other way, too: If a protocol has no counterexamples for, say, r = 100, we still might not know that it is really reliable; a counterexample might exist only when r is extremely large. Fortunately there are ways to obtain decent upper bounds on r, so that bounded model checking can be used to prove correctness as well as to demonstrate incorrectness. For example, we can verify the simplest known correct solution to Alice and Bob’s problem, a protocol by G. L. Peterson [Information Proc. Letters 12 (1981), 115– 116], who noticed that a careful combination of (43) and (45) actually suffices:

A0. Maybe go to A1.A1. Set a←1, go to A2.A2. Set  l←0, go to A3.A3.  Ifb⁢ go to A4, else to A5.A4. If l⁢ go to A5, else to A3.A5. Critical,  go to A6.A6. Set a←0, go to A0.B0. Maybe go to B1.B1. Set b←1, go to B2.B2. Set  l←1, go to B3.B3.  Ifa⁢ go to B4, else to B5.B4. If l⁢ go to B3, else to B5.B5. Critical,  go to B6.B6. Set a←0, go to B0.(49)

Now there are three signal lights, a, b, and l — one controlled by Alice, one controlled by Bob, and one switchable by both.

To show that states A5 and B5 can’t be concurrent, we can observe that the shortest counterexample will not repeat any state twice; in other words, it will be a simple path of transitions (33). Thus we can assume that r is at most the total number of states. However, (49) has 7 × 7 × 2 × 2 × 2 = 392 states; that’s a finite bound, not really out of reach for a good SAT solver on this particular problem, but we can do much better. For example, it’s not hard to devise clauses that are satisfiable if and only if there’s a simple path of length ≤ r (see exercise 92), and in this particular case the longest simple path turns out to have only 54 steps.

We can in fact do better yet by using the important notion of invariants, which we encountered in Section 1.2.1 and have seen repeatedly throughout this series of books. Invariant assertions are the key to most proofs of correctness, so it’s not surprising that they also give a significant boost to bounded model checking. Formally speaking, if Φ(X) is a Boolean function of the state vector X, we say that Φ is invariant if Φ(X) implies Φ(X′) whenever X → X′. For example, it’s not hard to see that the following clauses are invariant with respect to (49):

Φ(X)=(A0∨A1∨A2∨A3∨A4∨A5∨A6)∧(B0∨B1∨B2∨B3∨B4∨B5∨B6)∧(A0¯∨a¯)∧(A1¯∨a¯)∧(A2¯∨a)∧(A3¯∨a)∧(A4¯∨a)∧(A5¯∨a)∧(A6¯∨a)∧(B0¯∨b¯)∧(B1¯∨b¯)∧(B2¯∨b)∧(B3¯∨b)∧(B4¯∨b)∧(B5¯∨b)∧(B6¯∨b).(50)

(The clause A0¯∨a¯ says that a = 0 when Alice is in state A0, etc.) And we can use a SAT solver to prove that Φ is invariant, by showing that the clauses

Φ(X)∧(X→X′)∧¬Φ(X′)(51)

are unsatisfiable. Furthermore Φ(X_0) holds for the initial state X_0, because ¬Φ(X_0) is unsatisfiable. (See exercise 93.) Therefore Φ(Xt) is true for all t ≥ 0, by induction, and we may add these helpful clauses to all of our formulas.

The invariant (50) reduces the total number of states by a factor of 4. And the real clincher is the fact that the clauses

(X0→X1→...⁢ Xr)∧Φ(X0)∧Φ(X1)∧...∧Φ(Xr)∧A5r∧B5r,(52)

where X_0 is not required to be the initial state, turn out to be unsatisfiable when r = 3. In other words, there’s no way to go back more than two steps from a bad state, without violating the invariant. We can conclude that mutual exclusion needs to be verified for (49) only by considering paths of length 2(!). Furthermore, similar ideas (exercise 98) show that (49) is starvation-free.

Caveat: Although (49) is a correct protocol for mutual exclusion according to Alice and Bob’s ground rules, it cannot be used safely on most modern computers unless special care is taken to synchronize cache memories and write buffers. The reason is that hardware designers use all sorts of trickery to gain speed, and those tricks might allow one process to see a = 0 at time t + 1 even though another process has set a ← 1 at time t. We have developed the algorithms above by assuming a model of parallel computation that Leslie Lamport has called sequential consistency [IEEE Trans. C-28 (1979), 690–691].

Digital tomography

Another set of appealing questions amenable to SAT solving comes from the study of binary images for which partial information is given. Consider, for example, Fig. 79, which shows the “Cheshire cat” of Section 7.1.3 in a new light. This image is an m × n array of Boolean variables (xi,j), with m = 25 rows and n = 30 columns: The upper left corner element, x_1,1, is 0, representing white; and x_1,24 = 1 corresponds to the lone black pixel in the top row. We are given the row sums ri=Σj=1nxi,j for 1 ≤ i ≤ m and the column sums ci=Σi=1mxi,j for 1 ≤ j ≤ n, as well as both sets of sums in the 45° diagonal directions, namely
ad=Σi+j=d+1⁢     xi,j⁢      and  bd=Σi−j=d−n⁢     xi,j⁢              for  0<d<m+n.(53)

To what extent can such an image be reconstructed from its sums ri, cj, ad, and bd? Small examples are often uniquely determined by these Xray-like projections (see exercise 103). But the discrete nature of pixel images makes the reconstruction problem considerably more difficult than the corresponding continuous problem, in which projections from many different angles are available. Notice, for example, that the classical “8 queens problem” — to place eight nonattacking queens on a chessboard — is equivalent to solving an 8 × 8 digital tomography problem with the constraints ri = 1, cj = 1, ad ≤ 1, and bd ≤ 1.

Fig. 79. An array of black and white pixels together with its row sums ri, column sums cj, and diagonal sums ad, bd.

The constraints of Fig. 79 appear to be quite strict, so we might expect that most of the pixels xi,j are determined uniquely by the given sums. For instance, the fact that a_1 = ... = a_5 = 0 tells us that xi,j = 0 whenever i + j ≤ 6; and similar deductions are possible at all four corners of the image. A crude “ballpark estimate” suggests that we’re given a few more than 150 sums, most of which occupy 5 bits each; hence we have roughly 150 × 5 = 750 bits of data, from which we wish to reconstruct 25 × 30 = 750 pixels xi,j. Actually, however, this problem turns out to have many billions of solutions (see Fig. 80), most of which aren’t catlike! Exercise 106 provides a less crude estimate, which shows that this abundance of solutions isn’t really surprising.

Fig. 80. Extreme solutions to the constraints of Fig. 79.

A digital tomography problem such as Fig. 79 is readily represented as a sequence of clauses to be satisfied, because each of the individual requirements is just a special case of the cardinality constraints that we’ve already considered in the clauses of (18)–(21). This problem differs from the other instances of SAT that we’ve been discussing, primarily because it consists entirely of cardinality constraints: It is a question of solving 25 + 30 + 54 + 54 = 163 simultaneous linear equations in 750 variables xi,j, where each variable must be either 0 or 1. So it’s essentially an instance of integer programming (IP), not an instance of satisfiability (SAT). On the other hand, Bailleux and Boufkhad devised clauses (20) and (21) precisely because they wanted to apply SAT solvers, not IP solvers, to digital tomography. In the case of Fig. 79, their method yields approximately 40,000 clauses in 9,000 variables, containing about 100,000 literals altogether.

Figure 80(b) illustrates a solution that differs as much as possible from Fig. 79. Thus it minimizes the sum x_1,24 + x_2,5 + x_2,6 + ... + x_25,21 of the 182 variables that correspond to black pixels, over all 0-or-1-valued solutions to the linear equations. If we use linear programming to minimize that sum over 0 ≤ xi,j ≤ 1, without requiring the variables to be integers, we find almost instantly that the minimum value is ≈ 31.38 under these relaxed conditions; hence every black-and-white image must have at least 32 black pixels in common with Fig. 79. Furthermore, Fig. 80(b) — which can be computed in a few seconds by widely available IP solvers such as CPLEX — actually achieves this minimum. By contrast, state-of-the-art SAT solvers as of 2013 had great difficulty finding such an image, even when told that a 32-in-common solution is possible.

Parts (a) and (c) of Fig. 80 are, similarly, quite relevant to the current state of the SAT-solving art: They represent hundreds of individual SAT instances, where the first k variables are set to particular known values and we try to find a solution with the next variable either 0 or 1, respectively. Several of the subproblems that arose while computing rows 6 and 7 of Fig. 80(c) turned out to be quite challenging, although resolvable in a few hours; and similar problems, which correspond to different kinds of lexicographic order, apparently still lie beyond the reach of contemporary SAT-oriented methods. Yet IP solvers polish these problems off with ease. (See exercises 109 and 111.)

If we provide more information about an image, our chances of being able to reconstruct it uniquely are naturally enhanced. For example, suppose we also compute the numbers ri′,cj′,ad′, and bd′, which count the runs of 1s that occur in each row, column, and diagonal. (We have r1′=1,r2′=2,r3′=4, and so on.) Given this additional data, we can show that Fig. 79 is the only solution, because a suitable set of clauses turns out to be unsatisfiable. Exercise 117 explains one way by which (20) and (21) can be modified so that they provide constraints based on the run counts. Furthermore, it isn’t difficult to express even more detailed constraints, such as the assertion that “column 4 contains runs of respective lengths (6, 1, 3),” as a sequence of clauses; see exercise 438.

SAT examples — summary

We’ve now seen convincing evidence that simple Boolean clauses — ANDs of ORs of literals — are enormously versatile. Among other things, we’ve used them to encode problems of graph coloring, integer factorization, hardware fault testing, machine learning, model checking, and tomography. And indeed, Section 7.9 will demonstrate that 3SAT is the “poster child” for NP-complete problems in general: Any problem in NP — which is a huge class, essentially comprising all yes-or-no questions of size N whose affirmative answers are verifiable in NO(1) steps — can be formulated as an equivalent instance of 3SAT, without greatly increasing the problem size.

Backtracking for SAT

OK, we’ve seen a dizzying variety of intriguing and important examples of SAT that are begging to be solved. How shall we solve them?

Any instance of SAT that involves at least one variable can be solved systematically by choosing a variable and setting it to 0 or 1. Either of those choices gives us a smaller instance of SAT; so we can continue until reaching either an empty instance — which is trivially satisfiable, because no clauses need to be satisfied — or an instance that contains an empty clause. In the latter case we must back up and reconsider one of our earlier choices, proceeding in the same fashion until we either succeed or exhaust all the possibilities.

For example, consider again the formula F in (1). If we set x_1 = 0, F reduces to x¯2∧(x2∨x3), because the first clause (x1∨x¯2) loses its x_1, while the last two clauses contain x¯1 and are satisfied. It will be convenient to have a notation for this reduced problem; so let’s write

F|x¯1=x¯2∧(x2∨x3).(54)

Similarly, if we set x_1 = 1, we obtain the reduced problem

F|x1=(x2∨x3)∧x¯3∧(x¯2∨x3).(55)

F is satisfiable if and only if we can satisfy either (54) or (55).

In general if F is any set of clauses and if l is any literal, then F| l (read “F given l” or “F conditioned on l”) is the set of clauses obtained from F by

- removing every clause that contains l; and

- removing l¯ from every clause that contains l¯.

This conditioning operation is commutative, in the sense that F | l | l′ = F | l′ | l when l′≠l¯. If L = {l_1, ..., lk} is any set of strictly distinct literals, we can also write F | L = F |l_1 |...|lk. In these terms, F is satisfiable if and only if F |L = Ø for some such L, because the literals of L satisfy every clause of F when F |L = Ø.

The systematic strategy for SAT that was sketched above can therefore be formulated as the following recursive procedure B(F), which returns the special value ⊥ when F is unsatisfiable, otherwise it returns a set L that satisfies F :

B(F)={If F=∅, return ∅.⁢  (F⁢   is trivially satisfiable.)Otherwise if ϵ∈F, return ⊥.⁢  (F⁢  is unsatisfiable.)Otherwise let l⁢   be a liternal in F⁢  and set L⁢  ←B(F|l).If L≠⊥,return L∪l.⁢     Otherwise set L←B(F|l¯).If L≠⊥, return L∪l¯. Otherwise return ⊥.(56)

Let’s try to flesh out this abstract algorithm by converting it to efficient code at a lower level. From our previous experience with backtracking, we know that it will be crucial to have data structures that allow us to go quickly from F to F | l, then back again to F if necessary, when F is a set of clauses and l is a literal. In particular, we’ll want a good way to find all of the clauses that contain a given literal.

A combination of sequential and linked structures suggests itself for this purpose, based on our experience with exact cover problems: We can represent each clause as a set of cells, where each cell p contains a literal l = L(p) together with pointers F(p) and B(p) to other cells that contain l, in a doubly linked list. We’ll also need C(p), the number of the clause to which p belongs. The cells of clause Ci will be in consecutive locations START(i) + j, for 0 ≤ j <SIZE(i).

We will find it convenient to represent the literals xk and x¯k, which involve variable xk, by using the integers 2k and 2k + 1. With this convention we have
l¯=l⊕1    and   |l|=xl≫1.(57)

Our implementation of (56) will assume that the variables are x_1, x_2, ..., xn; thus the 2n possible literals will be in the range 2 ≤ l ≤ 2n + 1.

Cells 0 through 2n + 1 are reserved for special purposes: Cell l is the head of the list for the occurrences of l in other cells. Furthermore, if l is a literal whose value has not yet been fixed, C(l) will be the length of that list, namely the number of currently active clauses in which l appears.

For example, the m = 7 ternary clauses R′ of (7) might be represented internally in 2n + 2 + 3m = 31 cells as follows, using these conventions:

p=0123456789101112131415161718192021222324252627282930L(p)=−−−−−−−−−−973875653843862964742F(p)=−−30212917262822259738115615121341819162102320142724B(p)=−−24122015161113102514181928172352122273826309629742C(p)=−−23323332777666555444333222111

The literals of each clause appear in decreasing order here; for example, the literals L(p) = (8, 4, 3) in cells 19 through 21 represent the clause x4∨x2∨x¯1, which appears as the fourth clause, ‘41¯2’ in (7). This ordering turns out to be quite useful, because we’ll always choose the smallest unset variable as the l or l¯ in (56); then l or l¯ will always appear at the right of its clauses, and we can remove it or put it back by simply changing the relevant SIZE fields.

The clauses in this example have START(i) = 31 − 3i for 1 ≤ i ≤ 7, and SIZE(i) = 3 when computation begins.

Algorithm A (Satisfiability by backtracking). Given nonempty clauses C_1 ∧ ... ∧Cm on n > 0 Boolean variables x_1 ...xn, represented as above, this algorithm finds a solution if and only if the clauses are satisfiable. It records its current progress in an array m_1 ...mn of “moves,” whose significance is explained below.

A1. [Initialize.] Set a ← m and d ← 1. (Here a represents the number of active clauses, and d represents the depth-plus-one in an implicit search tree.)

A2. [Choose.] Set l ← 2d. If C(l) ≤ C(l + 1), set l ← l + 1. Then set md ← (l & 1) + 4[C(l ⊕ 1) = 0]. (See below.) Terminate successfully if C(l) = a.

A3. [Remove l¯.] Delete l¯ from all active clauses; but go to A5 if that would make a clause empty. (We want to ignore l¯, because we’re making l true.)

Fig. 81. The search tree that is implicitly traversed by Algorithm A, when that algorithm is applied to the eight unsatisfiable clauses R defined in (6). Branch nodes are labeled with the variable being tested; leaf nodes are labeled with a clause that is found to be contradicted.

A4. [Deactivate l’s clauses.] Suppress all clauses that contain l. (Those clauses are now satisfied.) Then set a ← a − C(l), d ← d + 1, and return to A2.

A5. [Try again.] If md < 2, set md ← 3 − md, l ← 2d +(md & 1), and go to A3.

A6. [Backtrack.] Terminate unsuccessfully if d = 1 (the clauses are unsatisfiable). Otherwise set d ← d − 1 and l ← 2d +(md & 1).

A7. [Reactivate l’s clauses.] Set a ← a + C(l), and unsuppress all clauses that contain l. (Those clauses are now unsatisfied, because l is no longer true.)

A8. [Unremove l¯.] Reinstate l¯ in all the active clauses that contain it. Then go back to A5.

(See exercise 121 for details of the low-level list processing operations that are needed to update the data structures in steps A3 and A4, and to downdate them in A7 and A8.)

The move codes mj of Algorithm A are integers between 0 and 5 that encode the state of the algorithm’s progress as follows:

- mj = 0 means we’re trying xj = 1 and haven’t yet tried xj = 0.

- mj = 1 means we’re trying xj = 0 and haven’t yet tried xj = 1.

- mj = 2 means we’re trying xj = 1 after xj = 0 has failed.

- mj = 3 means we’re trying xj = 0 after xj = 1 has failed.

- mj = 4 means we’re trying xj = 1 when x¯j doesn’t appear.

- mj = 5 means we’re trying xj = 0 when xj doesn’t appear.

Codes 4 and 5 refer to so-called “pure literals”: If no clause contains the literal l¯, we can’t go wrong by assuming that l is true.

For example, when Algorithm A is presented with the clauses (7), it cruises directly to a solution by setting m_1m_2m_3m_4 = 1014; the solution is x_1x_2x_3x_4 = 0101. But when the unsatisfiable clauses (6) are given, the successive code strings m_1 ...md in step A2 are

1,11,110,1131,121,1211,1221,21,211,2111,2121,221,2221,(58)

before the algorithm gives up. (See Fig. 81.)

It’s helpful to display the current string m_1 ...md now and then, as a convenient indication of progress; this string increases lexicographically. Indeed, fascinating patterns appear as the 2s and 3s gradually move to the left. (Try it!)

When the algorithm terminates successfully in step A2, a satisfying assignment can be read off from the move table by setting xj ← 1 ⊕ (mj & 1) for 1 ≤ j ≤ d. Algorithm A stops after finding a single solution; see exercise 122 if you want them all.

Lazy data structures

Instead of using the elaborate doubly linked machinery that underlies Algorithm A, we can actually get by with a much simpler scheme discovered by Cynthia A. Brown and Paul W. Purdom, Jr. [IEEE Trans. PAMI-4 (1982), 309–316], who introduced the notion of watched literals. They observed that we don’t really need to know all of the clauses that contain a given literal, because only one literal per clause is actually relevant at any particular time.

Here’s the idea: When we work on clauses F |L, the variables that occur in L have known values, but the other variables do not. For example, in Algorithm A, variable xj is implicitly known to be either true or false when j ≤ d, but its value is unknown when j > d. Such a situation is called a partial assignment. A partial assignment is consistent with a set of clauses if no clause consists entirely of false literals. Algorithms for SAT usually deal exclusively with consistent partial assignments; the goal is to convert them to consistent total assignments, by gradually eliminating the unknown values.

Thus every clause in a consistent partial assignment has at least one nonfalse literal; and we can adjust the data so that such a literal appears first, when the clause is represented in memory. Many nonfalse literals might be present, but only one of them is designated as the clause’s “watchee.” When a watched literal becomes false, we can find another nonfalse literal to swap into its place — unless the clause has been reduced to a unit, a clause of size 1.

With such a scheme we need only maintain a relatively short list for every literal l, namely a list Wl of all clauses that currently watch l. This list can be singly linked. Hence we need only one link per clause; and we have a total of only 2n + m links altogether, instead of the two links for each cell that are required by Algorithm A.

Furthermore — and this is the best part! — no updates need to be made to the watch lists when backtracking. The backtrack operations never falsify a nonfalse literal, because they only change values from known to unknown. Perhaps for this reason, data structures based on watched literals are called lazy, in contrast with the “eager” data structures of Algorithm A.

Let us therefore redesign Algorithm A and make it more laid-back. Our new data structure for each cell p has only one field, L(p); the other fields F(p), B(p), C(p) are no longer necessary, nor do we need 2n + 2 special cells. As before we will represent clauses sequentially, with the literals of Cj beginning at START(j) for 1 ≤ j ≤ m. The watched literal will be the one in START(j); and a new field, LINK(j), will be the number of another clause with the same watched literal (or 0, if Cj is the last such clause). Moreover, our new algorithm won’t need SIZE(j). Instead, we can assume that the final literal of Cj is in location START(j − 1) − 1, provided that we define START(0) appropriately.

The resulting procedure is almost unbelievably short and sweet. It’s surely the simplest SAT solver that can claim to be efficient on problems of modest size:

Algorithm B (Satisfiability by watching). Given nonempty clauses C_1 ∧... ∧Cm on n > 0 Boolean variables x_1 ... xn, represented as above, this algorithm finds a solution if and only if the clauses are satisfiable. It records its current progress in an array m_1 ... mn of “moves,” whose significance was explained above.

B1. [Initialize.] Set d ← 1.

B2. [Rejoice or choose.] If d > n, terminate successfully. Otherwise set md ← [W_2d = 0 or W_2d+1 ≠ 0] and l ← 2d + md.

B3. [Remove l¯ if possible.] For all j such that l¯ is watched in Cj, watch another literal of Cj. But go to B5 if that can’t be done. (See exercise 124.)

B4. [Advance.] Set Wl¯←0,d←d+1, and return to B2.

B5. [Try again.] If md < 2, set md ← 3 − md, l ← 2d +(md & 1), and go to B3.

B6. [Backtrack.] Terminate unsuccessfully if d = 1 (the clauses are unsatisfiable). Otherwise set d ← d − 1 and go back to B5.

Readers are strongly encouraged to work exercise 124, which spells out the low-level operations that are needed in step B3. Those operations accomplish essentially everything that Algorithm B needs to do.

This algorithm doesn’t use move codes 4 or 5, because lazy data structures don’t have enough information to identify pure literals. Fortunately pure literals are comparatively unimportant in practice; problems that are helped by the pure literal shortcut can usually also be solved quickly without it.

Notice that steps A2 and B2 use different criteria for deciding whether to try xd = 1 or xd = 0 first at each branch of the search tree. Algorithm A chooses the alternative that satisfies the most clauses; Algorithm B chooses to make l true instead of l¯ if the watch list for l¯ is empty but the watch list for l is not. (All clauses in which l¯ is watched will have to change, but those containing l are satisfied and in good shape.) In case of a tie, both algorithms set md ← 1, which corresponds to xd = 0. The reason is that human-designed instances of SAT tend to have solutions made up of mostly false literals.

Forced moves from unit clauses

The simple logic of Algorithm B works well on many problems that aren’t too large. But its insistence on setting x_1 first, then x_2, etc., makes it quite inefficient on many other problems, because it fails to take advantage of unit clauses. A unit clause (l) forces l to be true; therefore two-way branching is unnecessary whenever a unit clause is present. Furthermore, unit clauses aren’t rare: Far from it. Experience shows that they’re almost ubiquitous in practice, so that the actual search trees often involve only dozens of branch nodes instead of thousands or millions.

The importance of unit clauses was recognized already in the first computer implementation of a SAT solver, designed by Martin Davis, George Logemann, and Donald Loveland [CACM 5 (1962), 394–397] and based on ideas that Davis had developed earlier with Hilary Putnam [JACM 7 (1960), 201–215]. They extended Algorithm A by introducing mechanisms that recognize when the size of a clause decreases to 1, or when the number of unsatisfied clauses containing a literal drops to 0. In such cases, they put variables onto a “ready list,” and assigned those variables to fixed values before doing any further two-way branching. The resulting program was fairly complex; indeed, computer memory was so limited in those days, they implemented branching by writing all the data for the current node of the search tree onto magnetic tape, then backtracking when necessary by restoring the data from the most recently written tape records! The names of these four authors are now enshrined in the term “DPLL algorithm,” which refers generally to SAT solving via partial assignment and backtracking.

Brown and Purdom, in the paper cited earlier, showed that unit clauses can be detected more simply by using watched literals as in Algorithm B. We can supplement the data structures of that algorithm by introducing indices h_1 ...hn so that the variable whose value is being set at depth d is xhd instead of xd. Furthermore we can arrange the not-yet-set variables whose watch lists aren’t empty into a circular list called the “active ring”; the idea is to proceed through the active ring, checking to see whether any of its variables are currently in a unit clause. We resort to two-way branching only if we go all around the ring without finding any such units.

For example, let’s consider the 32 unsatisfiable clauses of waerden (3, 3; 9) in (9). The active ring is initially (1234567), because 8, 8¯, 9, and 9¯ aren’t being watched anywhere. There are no unit clauses yet. The algorithm below will decide to try 1¯ first; then it will change the clauses 123, 135, 147, and 159 to 213, 315, 417, and 519, respectively, so that nobody watches the false literal 1. The active ring becomes (234567) and the next choice is 2¯; so 213, 234, 246, and 258 morph respectively into 312, 324, 426, 528. Now, with active ring (34567), the unit clause ‘3’ is detected (because 1 and 2 are false in ‘312’). This precipitates further changes, and the first steps of the computation can be summarized thus:

Active ringx1x2x3x4x5x6x7x8UnitsChoiceChanged clauses(1234567)−−−−−−−−−1¯213,315,417,519(234567)0−−−−−−−−2¯312,324,426,528(34567)00−−−−−−−334¯3¯5¯,5¯3¯7¯,639¯(4567)001−−−−−−4¯624,714,546,648(567)0010−−−−−669¯3¯6¯,7¯6¯8¯(975)0010−1−−−9¯9¯(75)0010−1−−0778¯6¯7¯,8¯7¯9¯(85)0010−11−08¯8(5)0010−11005,5¯Backtrack(69785)001−−−−−−45¯3¯4¯,5¯4¯6¯,6¯4¯8¯(69785)0011−−−−−5¯5¯456,825,915,657,759(59)

When 6 is found, 7 is also a unit clause; but the algorithm doesn’t see it yet, because variable x_6 is tested first. The active ring changes first to (7 5) after 6 is found, because 5 is cyclically after 6; we want to look at 7 before 5, instead of revisiting more-or-less the same clauses. After 6 has been chosen, 9 is inserted at the left, because the watch list for 9¯ becomes nonempty. After backtracking, variables 8, 7, 9, 6 are successively inserted at the left as they lose their forced values.

Fig. 82. The search tree that is implicitly traversed by Algorithm D, when that algorithm is applied to the eight unsatisfiable clauses R defined in (6). Branch nodes are labeled with the variable being tested; leaf nodes are labeled with a clause that is found to be contradicted. When the right child of a branch node is a leaf, the left branch was forced by a conditional unary clause.

The following algorithm represents the active ring by giving a NEXT field to each variable, with xNEXT(k) the successor of xk. The ring is accessed via “head” and “tail” pointers h and t at the left and right, with h = NEXT(t). If the ring is empty, however, t = 0, and h is undefined.

Algorithm D (Satisfiability by cyclic DPLL). Given nonempty clauses C_1 ∧ ... ∧Cm on n > 0 Boolean variables x_1 ...xn, represented with lazy data structures and an active ring as explained above, this algorithm finds a solution if and only if the clauses are satisfiable. It records its current progress in an array h_1 ...hn of indices and an array m_0 ...mn of “moves,” whose significance is explained below.

D1. [Initialize.] Set m_0 ← d ← h ← t ← 0, and do the following for k = n, n−1, ... , 1: Set xk ← −1 (denoting an unset value); if W_2k ≠ 0 or W_2k+1 ≠ 0, set NEXT(k) ← h, h ← k, and if t = 0 also set t ← k. Finally, if t ≠ 0, complete the active ring by setting NEXT(t) ← h.

D2. [Success?] Terminate if t = 0 (all clauses are satisfied). Otherwise set k ← t.

D3. [Look for unit clauses.] Set h ← NEXT(k) and use the subroutine in exercise 129 to compute f ← [2h is a unit] + 2[2h + 1 is a unit]. If f = 3, go to D7. If f = 1 or 2, set md+1 ← f +3, t ← k, and go to D5. Otherwise, if h ≠ t, set k ← h and repeat this step.

D4. [Two-way branch.] Set h ← NEXT(t) and md+1 ← [W_2h =0 or W_2h+1 ≠0].

D5. [Move on.] Set d ← d +1, hd ← k ← h. If t = k, set t ← 0; otherwise delete variable k from the ring by setting NEXT(t) ← h ← NEXT(k).

D6. [Update watches.] Set b ← (md +1) mod 2, xk ← b, and clear the watch list for x¯k (see exercise 130). Return to D2.

D7. [Backtrack.] Set t ← k. While md ≥ 2, set k ← hd, xk ← −1; if W_2k ≠ 0 or W_2k+1 ≠ 0, set NEXT(k) ← h, h ← k, NEXT(t) ← h; and set d ← d − 1.

D8. [Failure?] If d > 0, set md ← 3 − md, k ← hd, and return to D6. Otherwise terminate the algorithm (because the clauses aren’t satisfiable).

The move codes of this algorithm are slightly different from the earlier ones:

- mj = 0 means we’re trying xhj=1 and haven’t yet tried xhj = 0.

- mj = 1 means we’re trying xhj=0 and haven’t yet tried xhj=1.

- mj = 2 means we’re trying xhj=1 after xhj=0 has failed.

- mj = 3 means we’re trying xhj=0 after xhj=1 has failed.

- mj = 4 means we’re trying xhj=1 because it’s forced by a unit clause.

- mj = 5 means we’re trying xhj=0 because it’s forced by a unit clause.

As before, the number of two-way branch nodes in the implicit search tree is the number of times that mj is set to 0 or 1.

Comparison of the algorithms. OK, we’ve just seen three rudimentary SAT solvers. How well do they actually do? Detailed performance statistics will be given later in this section, after we’ve studied several more algorithms. But a brief quantitative study of Algorithms A, B, and D now will give us some concrete facts with which we can calibrate our expectations before moving on.

Consider, for example, langford (n), the problem of Langford pairs. This problem is typical of SAT instances where many unit clauses arise during the computation. For example, when Algorithm D is applied to langford (5), it reaches a stage where the move codes are
m1m2...md=125555555555555114545545,(60)

indicating only four two-way branches (the 1s and the 2) amongst a sea of forced moves (the 4s and the 5s). We therefore expect Algorithm D to outperform Algorithms A and B, which don’t capitalize on unit clauses.

Sure enough, Algorithm D wins (slightly), even on a small example such as langford (5), which has 213 clauses, 480 cells, 28 variables. The detailed stats are

Algorithm A: 5379 + 108952 mems, 10552 bytes, 705 nodes.

Algorithm B: 1206 + 30789 mems, 4320 bytes, 771 nodes.

Algorithm D: 1417 + 28372 mems, 4589 bytes, 11 nodes.

(Here “5379+108952 mems” means that 5379 memory accesses were made while initializing the data structures before the algorithm began; then the algorithm itself accessed octabytes of memory 108,952 times.) Notice that Algorithm B is more than thrice as fast as Algorithm A in this example, although it makes 771 two-way branches instead of 705. Algorithm A needs fewer nodes, because it recognizes pure literals; but Algorithm B does much less work per node. Algorithm D, on the other hand, works very hard at each node, yet comes out ahead because its decision-making choices reduce the search to only a few nodes.

These differences become more dramatic when we consider larger problems. For instance, langford (9) has 1722 clauses, 3702 cells, 104 variables, and we find

Algorithm A: 332.0 megamems, 77216 bytes, 1,405,230 nodes.

Algorithm B: 53.4 megamems, 31104 bytes, 1,654,352 nodes.

Algorithm D: 23.4 megamems, 32057 bytes, 6093 nodes.

And with langford (13)’s 5875 clauses, 12356 cells, 228 variables, the results are

Algorithm A: 2699.1 gigamems, 253.9 kilobytes, 8.7 giganodes.

Algorithm B: 305.2 gigamems, 101.9 kilobytes, 10.6 giganodes.

Algorithm D: 71.7 gigamems, 104.0 kilobytes, 14.0 meganodes.

Mathematicians will recall that, at the beginning of Chapter 7, we used elementary reasoning to prove the unsatisfiability of langford (4k + 1) for all k. Evidently SAT solvers have great difficulty discovering this fact, even when k is fairly small. We are using that problem here as a benchmark test, not because we recommend replacing mathematics by brute force! Its unsatisfiability actually enhances its utility as a benchmark, because algorithms for satisfiability are more easily compared with respect to unsatisfiable instances: Extreme variations in performance occur when clauses are satisfiable, because solutions can be found purely by luck. Still, we might as well see what happens when our three algorithms are set loose on the satisfiable problem langford (16), which turns out to be “no sweat.” Its 11494 clauses, 23948 cells, and 352 variables lead to the statistics

Algorithm A: 11262.6 megamems, 489.2 kilobytes, 28.8 meganodes.

Algorithm B: 932.1 megamems, 196.2 kilobytes, 40.9 meganodes.

Algorithm D: 4.9 megamems, 199.4 kilobytes, 167 nodes.

Algorithm D is certainly our favorite so far, based on the langford data. But it is far from a panacea, because it loses badly to the lightweight Algorithm B on other problems. For example, the 2779 unsatisfiable clauses, 11662 cells, and 97 variables of waerden (3, 10; 97) yield

Algorithm A: 150.9 gigamems, 212.8 kilobytes, 106.7114 meganodes.

Algorithm B: 6.2 gigamems, 71.2 kilobytes, 106.7116 meganodes.

Algorithm D: 1430.4 gigamems, 72.1 kilobytes, 102.7 meganodes.

And waerden (3, 10; 96)’s 2721 satisfiable clauses, 11418 cells, 96 variables give us

Algorithm A: 96.9 megamems, 208.3 kilobytes, 72.9 kilonodes.

Algorithm B: 12.4 megamems, 69.8 kilobytes, 207.7 kilonodes.

Algorithm D: 57962.8 megamems, 70.6 kilobytes, 4447.7 kilonodes.

In such cases unit clauses don’t reduce the search tree size by very much, so we aren’t justified in spending so much time per node.

*Speeding up by working harder

Algorithms A, B, and D are OK on smallish problems, but they cannot really cope with the larger instances of SAT that have arisen in our examples. Significant enhancements are possible if we are willing to do more work and to develop more elaborate algorithms.

Mathematicians generally strive for nice, short, elegant proofs of theorems; and computer scientists generally aim for nice, short, elegant sequences of steps with which a problem can quickly be solved. But some theorems have no short proofs, and some problems cannot be solved efficiently with short programs.

Let us therefore adopt a new attitude, at least temporarily, by fearlessly deciding to throw lots of code at SAT: Let’s look at the bottlenecks that hinder Algorithm D on large problems, and let’s try to devise new methods that will streamline the calculations even though the resulting program might be ten times larger. In this subsection we shall examine an advanced SAT solver, Algorithm L, which is able to outperform Algorithm D by many orders of magnitude on many important problems. This algorithm cannot be described in just a few lines; but it does consist of cooperating procedures that are individually nice, short, elegant, and understandable by themselves.

The first important ingredient of Algorithm L is an improved mechanism for unit propagation. Algorithm D needs only a few lines of code in step D3 to discover whether or not the value of an unknown variable has been forced by previous assignments; but that mechanism isn’t particularly fast, because it is based on indirect inferences from a lazy data structure. We can do better by using “eager” data structures that are specifically designed to recognize forced values quickly, because high-speed propagation of the consequences of a newly asserted value turns out to be extremely important in practice.

A literal l is forced true when it appears in a clause C whose other literals have become false, namely when the set of currently assigned literals L has reduced C to the unit clause C |L = (l). Such unit clauses arise from the reduction of binary clauses. Algorithm L therefore keeps track of the binary clauses (u ∨ v) that are relevant to the current subproblem F | L. This information is kept in a so-called “bimp table” BIMP(l) for every literal l, which is a list of other literals l′ whose truth is implied by the truth of l. Indeed, instead of simply including binary clauses within the whole list of given clauses, as Algorithms A, B, and D do, Algorithm L stores the relevant facts about (u ∨ v) directly, in a ready-to-use way, by listing u in BIMP(υ¯) and v in BIMP(u¯). Each of the 2n tables BIMP(l) is represented internally as a sequential list of length BSIZE(l), with memory allocated dynamically via the buddy system (see exercise 134).

Binary clauses, in turn, are spawned by ternary clauses. For simplicity, Algorithm L assumes that all clauses have length 3 or less, because every instance of general SAT can readily be converted to 3SAT form (see exercise 28). And for speed, Algorithm L represents the ternary clauses by means of “timp tables,” which are analogous to the bimp tables: Every literal l has a sequential list TIMP(l) of length TSIZE(l), consisting of pairs p_1 = (u_1,v_1), p_2 = (u_2,v_2), ..., such that the truth of l implies that each (ui ∨ vi) becomes a relevant binary clause. If (u ∨ v ∨ w) is a ternary clause, there will be three pairs p = (v, w), p′ = (w, u), and p″ = (u, v), appearing in the respective lists TIMP(u¯), TIMP(υ¯), and TIMP(w¯). Moreover, these three pairs are linked together cyclically, with

Link(p)=p′,Link(p′)=p",Link(p")=p.(61)

Memory is allocated for the timp tables once and for all, as the clauses are input, because Algorithm L does not generate new ternaries during its computations. Individual pairs p are, however, swapped around within these sequential tables, so that the currently active ternary clauses containing u always appear in the first TSIZE(u¯) positions that have been allocated to TIMP(u¯).

For example, let’s consider again the ternary clauses (9) of waerden (3, 3; 9). Initially there are no binary clauses, so all BIMP tables are empty. Each of the ternary clauses appears in three of the TIMP tables. At level 0 of the search tree we might decide that x_5 = 0; then TIMP(5¯) tells us that we gain eight binary clauses, namely {13, 19, 28, 34, 37, 46, 67, 79}. These new binary clauses are represented by sixteen entries in BIMP tables; BIMP(3¯), for instance, will now be {1, 4, 7}. Furthermore, we’ll want all of the TIMP pairs that involve either 5 or 5¯ to become inactive, because the ternary clauses that contain 5 are weaker than the new binary clauses, and the ternary clauses that contain 5¯ are now satisfied. (See exercise 136.)

As in (57) above, we shall assume that the variables of a given formula are numbered from 1 to n, and we represent the literals k and k¯ internally by the numbers 2k and 2k+1. Algorithm L introduces a new twist, however, by allowing variables to have many different degrees of truth [see M. Heule, M. Dufour, J. van Zwieten, and H. van Maaren, LNCS 3542 (2005), 345–359]: We say that xk is true with degree D if VAL[k] = D, and false with degree D if VAL[k] = D + 1, where D is any even number.

The highest possible degree, typically 2^32 − 2 inside a computer, is called RT for “real truth.” The next highest degree, typically 2^32 − 4, is called NT for “near truth”; and then comes PT = 2^32 − 6, “proto truth.” Lower degrees PT − 2, PT − 4, ..., 2 also turn out to be useful. A literal l is said to be fixed in context T if and only if VAL[|l|] ≥ T; it is fixed true if we also have VAL[|l|] &1 = l &1, and it is fixed false if its complement l¯ is fixed true.

Suppose, for example, that VAL[2] = RT + 1 and VAL[7] = PT; hence x_2 is “really false” while x_7 is “proto true.” Then the literal ‘7’, represented internally by l = 14, is fixed true in context PT, but l is not fixed in contexts NT or RT. The literal ‘2¯’, represented internally by l = 5, is fixed true in every context.

Algorithm L uses a sequential stack R_0, R_1, ..., to record the names of literals that have received values. The current stack size, E, satisfies 0 ≤ E ≤ n. With those data structures we can use a simple breadth-first search procedure to propagate the binary consequences of a literal l in context T at high speed:

Set H←E; take account of l;while H<E,set l←RH,H←H+1, and(62)take account of l′⁢   for all l′⁢  in BIMP(l).

Here “take account of l” means “if l is fixed true in context T, do nothing; if l is fixed false in context T , go to step CONFLICT; otherwise set VAL[|l|] ← T +(l&1), RE ← l, and E ← E + 1.” The step called CONFLICT is changeable.

A literal’s BIMP table might grow repeatedly as computation proceeds. But we can undo the consequences of bad decisions by simply resetting BSIZE(l) to the value that it had before those decisions were made. A special variable ISTAMP is increased whenever we begin a new round of decision-making, and each literal l has its private stamp IST(l). Whenever BSIZE(l) is about to increase, we check if IST(l) = ISTAMP. If not, we set

IST(l)←ISTAMP,ISTACK[I]←(l,BSIZE(l)),I←I+1.(63)

Then the entries on ISTACK make it easy to downdate the BIMP tables when we backtrack. (See step L13 in the algorithm below.)

We’re ready now to look at the detailed steps of Algorithm L, except that one more member of its arsenal of data structures needs to be introduced: There’s an array VAR, which contains a permutation of {1, ..., n}, with VAR[k] = x if and only if INX[x] = k. Furthermore VAR[k] is a “free variable” — not fixed in context RT — if and only if 0 ≤ k < N. This setup makes it convenient to keep track of the variables that are currently free: A variable becomes fixed by swapping it to the end of the free list and decreasing N (see exercise 137); then we can free it later by simply increasing N, without swapping.

Algorithm L (Satisfiability by DPLL with lookahead). Given nonempty clauses C_1 ∧ ... ∧ Cm of size ≤ 3, on n > 0 Boolean variables x_1 ...xn, this algorithm finds a solution if and only if the clauses are satisfiable. Its family of cooperating data structures is discussed in the text.

L1. [Initialize.] Record all binary clauses in the BIMP array and all ternary clauses in the TIMP array. Let U be the number of distinct variables in unit clauses; terminate unsuccessfully if two unit clauses contradict each other, otherwise record all distinct unit literals in FORCE[k] for 0 ≤ k < U. Set VAR[k] ← k + 1 and INX[k + 1] ← k for 0 ≤ k < n; and d ← F ← I ← ISTAMP ← 0. (Think d = depth, F = fixed variables, I = ISTACK size.)

L2. [New node.] Set BRANCH[d] ← −1. If U = 0, invoke Algorithm X below (which looks ahead for simplifications and also gathers data about how to make the next branch). Terminate happily if Algorithm X finds all clauses satisfied; go to L15 if Algorithm X discovers a conflict; go to L5 if U > 0.

L3. [Choose l.] Select a literal l that’s desirable for branching (see exercise 168). If l = 0, set d ← d + 1 and return to L2. Otherwise set DEC[d] ← l, BACKF[d] ← F, BACKI[d] ← I, and BRANCH[d] ← 0.

L4. [Try l.] Set U ← 1, FORCE[0] ← l.

L5. [Accept near truths.] Set T ← NT, G ← E ← F, ISTAMP ← ISTAMP +1, and CONFLICT ← L11. Perform the binary propagation routine (62) for l ← FORCE[0], ..., l ← FORCE[U − 1]; then set U ← 0.

L6. [Choose a nearly true L.] (At this point the stacked literals Rk are “really true” for 0 ≤ k < G, and “nearly true” for G ≤ k < E. We want them all to be really true.) If G = E, go to L10. Otherwise set L ← RG, G ← G+1.

L7. [Promote L to real truth.] Set X ← |L| and VAL[X] ← RT + L&1. Remove variable X from the free list and from all TIMP pairs (see exercise 137). Do step L8 for all pairs (u, v) in TIMP(L), then return to L6.

L8. [Consider u ∨ v.] (We have deduced that u or v must be true; five cases arise.) If either u or v is fixed true (in context T , which equals NT), do nothing. If both u and v are fixed false, go to CONFLICT. If u is fixed false but v isn’t fixed, perform (62) with l ← v. If v is fixed false but u isn’t fixed, perform (62) with l ← u. If neither u nor v is fixed, do step L9.

L9. [Exploit u ∨ v.] If v¯∈BIMP(u¯), perform (62) with l ← u (because u¯ implies both v and υ¯). Otherwise if v∈BIMP(u¯), do nothing (because we already have the clause u ∨ ν). Otherwise if u¯∈BIMP(v¯), perform (62) with l ← v. Otherwise append v to BIMP(u¯) and u to BIMP(υ¯). (Each change to BIMP means that (63) might be invoked. Exercise 139 explains how to improve this step by deducing further implications called “compensation resolvents.”)

L10. [Accept real truths.] Set F ← E. If BRANCH[d] ≥ 0, set d ← d + 1 and go to L2. Otherwise go to L3 if d > 0, to L2 if d = 0.

L11. [Unfix near truths.] While E > G, set E ← E − 1 and VAL[|RE|] ← 0.

L12. [Unfix real truths.] While E > F, do the following: Set E ← E − 1 and X ← |RE|; reactivate the TIMP pairs that involve X and restore X to the free list (see exercise 137); set VAL[X] ← 0.

L13. [Downdate BIMPs.] If BRANCH[d] ≥ 0, do the following while I > BACKI[d]: Set I ← I − 1 and BSIZE(l) ← s, where ISTACK[I] = (l, s).

L14. [Try again?] (We’ve discovered that DEC[d] doesn’t work.) If BRANCH[d] = 0, set l ← DEC[d], DEC[d] ← l ← l¯, BRANCH[d] ← 1, and go back to L4.

L15. [Backtrack.] Terminate unsuccessfully if d = 0. Otherwise set d ← d − 1, E ← F, F ← BACKF[d], and return to L12.

Exercise 143 extends this algorithm so that it will handle clauses of arbitrary size.

*Speeding up by looking ahead

Algorithm L as it stands is incomplete, because step L2 relies on an as-yet-unspecified “Algorithm X” before it chooses a literal for branching. If we use the simplest possible Algorithm X, by branching on whatever literal happens to be first in the current list of free variables, the streamlined methods for propagating forced moves in (62) and (63) will tend to make Algorithm L run roughly three times as fast as Algorithm D, and that isn’t a negligible improvement. But with a sophisticated Algorithm X we can often gain another factor of 10 or more in speed, on significant problems.

For example, here are some typical empirical statistics:

Problem

Algorithm D

Algorithm L^0

Algorithm L+

waerden (3, 10; 97)

1430 gigamems,
103 meganodes

391 gigamems,
31 meganodes

772 megamems,
4672 nodes

langford (13)

71.7 gigamems,
14.0 meganodes

21.5 gigamems,
10.9 meganodes

45.7 gigamems,
944 kilonodes

rand (3, 420, 100, 0)

184 megamems,
34 kilonodes

34 megamems,
7489 nodes

626 kilomems,
19 nodes

Here Algorithm L^0 stands for Algorithm L with the simplest Algorithm X, while Algorithm L+ uses all of the lookahead heuristics that we are about to discuss. The first two problems involve rather large clauses, so they use the extended Algorithm L of exercise 143. The third problem consists of 420 random ternary clauses on 100 variables. (Algorithm B, incidentally, needs 80.1 teramems, and a search tree of 4.50 teranodes, to show that those clauses are unsatisfiable.)

The moral of this story is that it’s wise to do 100 times as much computation at every node of a large search tree, if we can thereby decrease the size of the tree by a factor of 1000.

How then can we distinguish a variable that’s good for branching from a variable that isn’t? We shall consider a three-step approach:

- Preselecting, to identify free variables that appear to be good candidates;

- Nesting, to allow candidate literals to share implied computations;

- Exploring, to examine the immediate consequences of hypothetical decisions.

While carrying out these steps, Algorithm X might discover a contradiction (in which case Algorithm L will take charge again at step L15); or the lookahead process might discover that several of the free literals are forced to be true (in which case it places them in the first U positions of the FORCE array). The explorations might even discover a way to satisfy all of the clauses (in which case Algorithm L will terminate and everybody will be happy). Thus, Algorithm X might do much more than simply choose a good variable on which to branch.

The following recommendations for Algorithm X are based on Marijn Heule’s lookahead solver called march, one of the world’s best, as it existed in 2013.

The first stage, preselection, is conceptually simplest, although it also involves some “handwaving” because it depends on necessarily shaky assumptions. Suppose there are N free variables. Experience has shown that we tend to get a good heuristic score h(l) for each literal l, representing the relative amount by which asserting l will reduce the current problem, if these scores approximately satisfy the simultaneous nonlinear equations
h(l)=0.1+α⁢Σu∈BIMP(l)u⁢   ⁢not fixedh^(u)+Σ(u,υ)∈TIMP(l)h^(u)h^(ν).(64)

Here α is a magic constant, typically 3.5; and h^(l) is a multiple of h(l) chosen so that Σl⁢  h^(l)=2N is the total number of free literals. (In other words, the h scores on the right are “normalized” so that their average is 1.)

Any given set of scores h(l) can be used to derive a refined set h′(l) by letting
h′(l)=0.1+αΣu∈BIMP(l)u⁢  not fixed h(u)have+Σ(u,υ)∈TIMP(l)h(u)haveh(υ)have,  have=12NΣlh(l).(65)

Near the root of the search tree, when d ≤ 1, we start with h(l) = 1 for all l and then refine it five times (say). At deeper levels we start with the h(l) values from the parent node and refine them once. Exercise 145 contains an example.

We’ve computed h(l) for all of the free literals l, but we won’t have time to explore them all. The next step is to select free variables CAND[j] for 0 ≤ j < C, where C isn’t too large; we will insist that the number of candidates does not exceed

Cmax=max(C0,C1/d),(66)

using cutoff parameters that are typically C_0 = 30, C_1 = 600. (See exercise 148.)

We start by dividing the free variables into “participants” and “newbies”: A participant is a variable such that either x or x¯ has played the role of u or v in step L8, at some node above us in the search tree; a newbie is a nonparticipant. When d = 0 every variable is a newbie, because we’re at the root of the tree. But usually there is at least one participant, and we want to branch only on participants whenever possible, in order to maintain focus while backtracking.

If we’ve got too many potential candidates, even after restricting consideration to participants, we can winnow the list down by preferring the variables x that have the largest combined score h(x)h(x¯). Step X3 below describes a fairly fast way to come up with the desired selection of C ≤ Cmax candidates.

A simple lookahead algorithm can now proceed to compute a more accurate heuristic score H(l), for each of the 2C literals l = CAND[j] or l = ¬CAND[j] that we’ve selected for further scrutiny. The idea is to simulate what would happen if l were used for branching, by mimicking steps L4–L9 (at least to a first approximation): Unit literals are propagated as in the exact algorithm, but whenever we get to the part of step L9 that changes the BIMP tables, we don’t actually make such a change; we simply note that a branch on l would imply u ∨ v, and we consider the value of that potential new clause to be h(u)h(v). The heuristic score H(l) is then defined to be the sum of all such clause weights:

H(l)=Σ{h(u)h(υ)   |   asserting l⁢  in L4 leads to asserting u⁢   ∨υ⁢   in L9}.(67)

For example, the problem waerden (3, 3; 9) of (9) has nine candidate variables {1, 2, ..., 9} at the root of the search tree, and exercise 145 finds their rough heuristic scores h(l). The more discriminating scores H(l) turn out to be

H(1)=h(2¯)h(3¯)+h(3¯)h(5¯)+h(4¯)h(7¯)+h(5¯)h(9¯)≈168.6;H(2)=h(1¯)h(3¯)+h(3¯)h(4¯)+h(4¯)h(6¯)+h(5¯)h(8¯)≈157.3;H(3)=h(1¯)h(2¯)+h(2¯)h(4¯)+h(4¯)h(5¯)+...+h(6¯)h(9¯)≈233.4;H(4)=h(2¯)h(3¯)+h(3¯)h(5¯)+h(5¯)h(6¯)+...+h(1¯)h(7¯)≈231.8;H(5)=h(3¯)h(4¯)+h(4¯)h(6¯)+h(6¯)h(7¯)+...+h(1¯)h(9¯)≈284.0.

This problem is symmetrical, so we also have H(6)=H(6¯)=H(4)=H(4¯), etc. The best literal for branching, according to this estimate, is 5 or 5¯.

Suppose we set x_5 false and proceed to look ahead at the reduced problem, with d = 1. At this point there are eight candidates, {1, 2, 3, 4, 6, 7, 8, 9}; and they’re now related also by binary implications, because the original clause ‘357’ has, for instance, been reduced to ‘37’. In fact, the BIMP tables now define the dependency digraph

because 3¯→7, etc.; and in general the 2C candidate literals will define a dependency digraph whose structure yields important clues about the current subproblem. We can, for example, use Tarjan’s algorithm to find the strong components of that digraph, as mentioned after Theorem 7.1.1K. If some strong component includes both l and l¯, the current subproblem is unsatisfiable. Otherwise two literals of the same component are constrained to have the same value; so we shall choose one literal from each of the S ≤ 2C strong components, and use those choices as the actual candidates for lookahead.

Continuing our example, at this point we can use a nice trick to save redundant computation, by extracting a subforest of the dependency digraph:

The relation 8¯→2 means that whatever happens after asserting the literal ‘2’ will also happen after asserting ‘8¯’; hence we need not repeat the steps for ‘2’ while studying ‘8¯’. And similarly, each of the other subordinate literals ‘1¯’, ..., ‘9¯’ inherits the assertions of its parent in this hierarchy. Tarjan’s algorithm actually produces such a subforest with comparatively little extra work.

The nested structure of a forest also fits beautifully with “degrees of truth” in our data structure, if we visit the S candidate literals in preorder of the subforest, and if we successively assert each literal l at the truth degree that corresponds to twice its position in postorder. For instance, (69) becomes the following arrangement, which we shall call the “lookahead forest”:

preorder128¯3464¯73¯6¯9¯82¯91¯7¯(70)2⋅postorder2648101412221618202624322830

A simulation of steps L4–L9 with l ← 1 and T ← 2 makes x_1 true at degree 2 (we say that it’s “2fixed” or “2true”); it also computes the score H(1)←h(2¯)h(3¯)+h(4¯)h(7¯), but spawns no other activity if Algorithm Y below isn’t active. Simulation with l ← 2 and T ← 6 then 6fixes 2 and computes H(2)←h(1¯)h(3¯)+h(3¯)h(4¯)+h(4¯)h(6¯); during this process x_1’s value isn’t seen, because it is less than T . Things get interesting when l←8¯ and T ← 4: Now we 4fix 8¯, and we’re still able to see that x_2 is true because 6 > T. So we save a little computation by inheriting H(2) and setting H(8¯)←H(2)+h(4)h(6)+h(6)h(7)+h(7)h(9).

The real action begins to break through a few steps later, when we set l←4¯ and T ← 12. Then (62) will 12fix not only 4¯ but also 3, since 4¯→3; and the 12truth of 3 will soon take us to the simulated step L8 with u=6¯ and υ=9. Aha: We 12fix 9¯, because 6 is 14true. Then we also 12fix the literals 7, 1, ... , and reach a contradiction. This contradiction shows that branching on 4¯ will lead to a conflict; hence the literal 4 must be true, if the current clauses are satisfiable.

Whenever the lookahead simulation of Algorithm X learns that some literal l must be true, as in this example, it places l on the FORCE list and makes l proto true (that is, true in context PT). A proto true literal will remain fixed true throughout this round of lookahead, because all relevant values of T will be less than PT. Later, Algorithm L will promote proto truth to near truth, and ultimately to real truth — unless a contradiction arises. (And in the case of waerden (3, 3; 9), such a contradiction does in fact arise; see exercise 150.)

Why does the combination of preorder and postorder work so magically in (70)? It’s because of a basic property of forests in general, which we noted for example in exercise 2.3.2–20: If u and v are nodes of a forest, u is a proper ancestor of v if and only if u precedes v in preorder and u follows v in postorder. Moreover, when we look ahead at candidate literals in this way, an important invariant relation is maintained on the R stack, namely that truth degrees never increase as we move from the bottom to the top:

VAL[|Rj−1|]|1⁢  ≥  VAL[|Rj|],for  1 ≤j<E.(71)

Real truths appear at the bottom, then near truths, then proto truths, etc. For example, the stack at one point in the problem above contains seven literals,
j=0123456Rj=5¯64¯39¯71VAL[|Rj|]=RT+1141312131212

One consequence is that the current visibility of truth values matches the recursive structure by which false literals are purged from ternary clauses.

The second phase of Algorithm X, after preselection of candidates, is called “nesting,” because it constructs a lookahead forest analogous to (70). More precisely, it constructs a sequence of literals LL[j] and corresponding truth offsets LO[j], for 0 ≤ j < S. It also sets up PARENT pointers to indicate the forest structure more directly; for example, with (69) we would have PARENT(8¯)=2 and PARENT(2) = ∧.

The third phase, “exploration,” now does the real work. It uses the lookahead forest to evaluate heuristics H(l) for the candidate literals — and also (if it’s lucky) to discover literals whose values are forced.

The heart of the exploration phase is a breadth-first search based on steps L5, L6, and L8. This routine propagates truth values of degree T and also computes w, the weight of new binary clauses that would be spawned by branching on l:

Set l0←l,i←w←0, and G←E←F;  perform  (62);while G<E, set L←RG,G←G+1, andtake account of (u,υ) for all (u,υ) in TIMP(L);(72)generate new binary clauses (l¯0∨Wk) for 0≤k<i.

Here “take account of (u, v)” means “if either u or v is fixed true (in context T ), do nothing; if both u and v are fixed false, go to CONFLICT; if u is fixed false but v isn’t fixed, set Wi ← v, i ← i + 1, and perform (62) with l ← v; if v is fixed false but u isn’t fixed, set Wi ← u, i ← i + 1, and perform (62) with l ← u; if neither u nor v is fixed, set w ← w + h(u)h(v).”

Explanation: A ternary clause of the form l¯ ∨ u ∨ v, where L is fixed true and u is fixed false as a consequence of l_0 being fixed true, is called a “windfall.” Such clauses are good news, because they imply that the binary clause l¯0∨υ must be satisfied in the current subproblem. Windfalls are recorded on a stack called W, and appended to the BIMP database at the end of (72).

The exploration phase also exploits an important fact called the autarky principle, which generalizes the notion of “pure literal” that we discussed above in connection with Algorithm A. An “autarky” for a SAT problem F is a set of strictly distinct literals A = {a_1, ..., at} with the property that every clause of F either contains at least one literal of A or contains none of the literals of A¯={a¯1,...,a¯2}. In other words, A satisfies every clause that A or A¯ “touches.”

An autarky is a self-sufficient system. Whenever A is an autarky, we can assume without loss of generality that all of its literals are actually true; for if F is satisfiable, the untouched clauses are satisfiable, and A tells us how to satisfy the touched ones. Step X9 of the following algorithm shows that we can detect certain autarkies easily while we’re looking ahead.

Algorithm X (Lookahead for Algorithm L). This algorithm, which is invoked in step L2 of Algorithm L, uses the data structures of that algorithm together with additional arrays of its own to explore properties of the current subproblem. It discovers U ≥ 0 literals whose values are forced, and puts them in the FORCE array. It terminates either by (i) satisfying all clauses; (ii) finding a contradiction; or (iii) computing heuristic scores H(l) that will allow step L3 to choose a good literal for branching. In case (iii) it might also discover new binary clauses.

X1. [Satisfied?] If F = n, terminate happily (no variables are free).

X2. [Compile rough heuristics.] Set N = n − F . For each free literal l, set VAL[l] ← 0, and use (65) to compute a rough score h(l).

X3. [Preselect candidates.] Let C be the current number of free variables that are “participants,” and put them into the CAND array. If C = 0, set C ← N and put all free variables into CAND; terminate happily, however, if all clauses are satisfied (see exercise 152). Give each variable x in CAND the rating r(x)=h(x)h(x¯). Then while C > 2Cmax (see (66)), delete all elements of CAND whose rating is less than the mean rating; but terminate this loop if no elements are actually deleted. Finally, if C > Cmax, reduce C to Cmax by retaining only top-ranked candidates. (See exercise 153.)

X4. [Nest the candidates.] Construct a lookahead forest, represented in LL[j] and LO[j] for 0 ≤ j < S and by PARENT pointers (see exercise 155).

X5. [Prepare to explore.] Set U′ ← j′ BASE ← j ← 0 and CONFLICT ← X13.

X6. [Choose l for lookahead.] Set l ← LL[j] and T ← BASE + LO[j]. Set H(l) ← H(PARENT(l)), where H(∧) = 0. If l is not fixed in context T, goto X8. Otherwise, if l is fixed false but not proto false, do step X12 with l←l¯.

X7. [Move to next.] If U > U′, set U′ ← U and j′ ← j. Then set j ← j + 1. If j = S, set j ← 0 and BASE ← BASE + 2S. Terminate normally if j = j′, or if j = 0 and BASE + 2S ≥ PT (beware of overflow). Otherwise return to X6.

X8. [Compute sharper heuristic.] Perform (72). Then if w > 0, set H(l_0) ← H(l_0) + w and go to X10.

X9. [Exploit an autarky.] If H(l_0) = 0, do step X12 with l ← l_0. Otherwise generate the new binary clause l_0 ∨ ¬PARENT(l_0). (Exercise 166 explains why.)

X10. [Optionally look deeper.] Perform Algorithm Y below.

X11. [Exploit necessary assignments.] Do step X12 for all literals l ∈ BIMP(l¯0) that are fixed true but not proto true. Then go to X7. (See exercise 167.)

X12. [Force l.] Set FORCE[U] ← l, U ← U + 1, T′ ← T , and perform (72) with T ← PT. Then set T ← T′. (This step is a subroutine, used by other steps.)

X13. [Recover from conflict.] If T < PT, do step X12 with l←l¯0 and go to X7. Otherwise terminate with a contradiction.

Notice that, in steps X5–X7, this algorithm proceeds cyclically through the forest, continuing to look ahead until completing a pass in which no new forced literals are found. The BASE address of truth values continues to grow, if necessary, but it isn’t allowed to become too close to PT.

*Looking even further ahead

If it’s a good idea to look one step ahead, maybe it’s a better idea to look two steps ahead. Of course that’s a somewhat scary proposition, because our data structures are already pretty stretched; and besides, double lookahead might take way too much time. Nevertheless, there’s a way to pull it off, and to make Algorithm L run even faster on many problems.

Algorithm X looks at the immediate consequences of assuming that some literal l_0 is true. Algorithm Y, which is launched in step X10, goes further out on that limb, and investigates what would happen if another literal, l^0, were also true. The goal is to detect branches that die off early, allowing us to discover new implications of l_0 or even to conclude that l_0 must be false.

For this purpose Algorithm Y stakes out an area of truth space between the current context T and a degree of truth called “double truth” or DT, which is defined in step Y2. The size of this area is determined by a parameter Y, which is typically less than 10. The same lookahead forest is used to give relative truth degrees below DT. Double truth is less trustworthy than proto truth, PT; but literals that are fixed at level DT are known to be conditionally true (“Dtrue”) or conditionally false (“Dfalse”) under the hypothesis that l_0 is true.

Going back to our example of waerden (3, 3; 9), the scenario described above was based on the assumption that double lookahead was not done. Actually, however, further activity by Algorithm Y will usually take place after H(1) has been set to h(2¯)h(3¯)+h(4¯)h(7¯). The value of DT will be set to 130, assuming that Y = 8, because S = 8. Literal 1 will become Dtrue. Looking then at 2 will 6fix 2; and that will 6fix 3¯ because of the clause 1¯2¯3¯. Then 3¯ will 6fix 4 and 7, contradicting 1¯4¯7¯ and causing 2 to become Dfalse. Other literals also will soon become Dtrue or Dfalse, leading to a contradiction; and that contradiction will allow Algorithm Y to make literal 1 proto false before Algorithm X has even begun to look ahead at literal 2.

The main loop of double lookahead is analogous to (72), but it’s simpler, because we’re further removed from reality:

Set l^0←l and G←G←E←F;  perform  (62);while G<E, set L←RG,G←G+1, and(73)take account of (u,υ) for all (u,υ) in TIMP(L);

Now “take account of (u, v)” means “if either u or v is fixed true (in context T ), or if neither u nor v is fixed, do nothing; if both u and v are fixed false, go to CONFLICT; if u is fixed false but v isn’t fixed, perform (62) with l ← v; if v is fixed false but u isn’t fixed, perform (62) with l ← u.”

Since double-looking is costly, we want to try it only when there’s a fairly good chance that it will be helpful, namely when H(l_0) is large. But how large is large enough? The proper threshold depends on the problem being solved: Some sets of clauses are handled more quickly by double-looking, while others are immune to such insights. Marijn Heule and Hans van Maaren [LNCS 4501 (2007), 258–271] have developed an elegant feedback mechanism that automatically tunes itself to the characteristics of the problem at hand: Let τ be a “trigger,” initially 0. Step Y1 allows double-look only if H(l_0) > τ; otherwise τ is decreased to βτ, where β is a damping factor (typically 0.999), so that double-looking will become more attractive. On the other hand if double-look doesn’t find a contradiction that makes l_0 proto false, the trigger is raised to H(l_0) in step Y6.

Algorithm Y (Double lookahead for Algorithm X). This algorithm, invoked in step X10, uses the same data structures (and a few more) to look ahead more deeply. Parameters β and Y are explained above. Initially DFAIL(l) = 0 for all l.

Y1. [Filter.] Terminate if DFAIL(l_0) = ISTAMP, or if T + 2S(Y + 1) > PT. Otherwise, if H(l_0) ≤ τ, set τ ← βτ and terminate.

Y2. [Initialize.] Set BASE ← T − 2, LBASE ← BASE + 2S·Y, DT ← LBASE + LO[j], i←j^′←j^←0, E ← F, and CONFLICT ← Y8. Perform (62) with l ← l_0 and T ← DT.

Y3. [Choose l for double look.] Set l ← LL[j^] and T ← BASE + LO[j^]. If l is not fixed in context T, go to Y5. Otherwise, if l is fixed false but not Dfalse, do step Y7 with l←l^.

Y4. [Move to next.] Set j^ ← j^ + 1. If j^ = S, set j^ ← 0 and BASE ← BASE + 2S. Go to Y6 if j^′ = j^, or if j^ = 0 and BASE = LBASE. Otherwise return to Y3.

Y5. [Look ahead.] Perform (73), and return to Y4 (if no conflict arises).

Y6. [Finish.] Generate new binary clauses (l¯0∨Wk) for 0 ≤ k < i. Then set BASE ← LBASE, T ← DT, τ ← H(l_0), DFAIL(l_0) ← ISTAMP, CONFLICT ← X13, and terminate.

Y7. [Assume also l.] Set j^′←j^, T′ T, and perform (73) with T ← DT. Then set T ← T′, Wi←l^0, i ← i + 1. (This step is a subroutine.)

Y8. [Recover from conflict.] If T < DT, do step Y7 with l ← ¬LL[j^] and go to Y4. Otherwise set BASE ← LBASE, CONFLICT ← X13, and exit to X13.

Some quantitative statistics will help to ground these algorithms in reality: When Algorithm L was let loose on rand (3, 2062, 500, 314), a problem with 500 variables and 2062 random ternary clauses, it proved unsatisfiability after making 684,433,234,661 memory accesses and constructing a search tree of 9,530,489 nodes. Exercise 173 explains what would have happened if various parts of the algorithm had been disabled. None of the other SAT solvers we shall discuss are able to handle such large random problems in a reasonable amount of time.

Random satisfiability

There seems to be no easy way to analyze the satisfiability problem under random conditions. In fact, the basic question “How many random clauses of 3SAT on n variables do we need to consider, on the average, before they can’t all be satisfied?” is a famous unsolved research problem.

From a practical standpoint this question isn’t as relevant as the analogous questions were when we studied algorithms for sorting or searching, because real-world instances of 3SAT tend to have highly nonrandom clauses. Deviations from randomness in combinatorial algorithms often have a dramatic effect on running time, while methods of sorting and searching generally stay reasonably close to their expected behavior. Thus a focus on randomness can be misleading. On the other hand, random SAT clauses do serve as a nice, clean model, so they give us insights into what goes on in Boolean territory. Furthermore the mathematical issues are of great interest in their own right. And fortunately, much of the basic theory is in fact elementary and easy to understand. So let’s take a look at it.

Exercise 180 shows that random satisfiability can be analyzed exactly, when there are at most five variables. We might as well start there, because the “tiny” 5-variable case is still large enough to shed some light on the bigger picture. When there are n variables and k literals per clause, the number N of possible clauses that involve k different variables is clearly 2k(nk): There are (nk) ways to choose the variables, and 2k ways to either complement or not. So we have, for example, N=23(53)=80 possible clauses in a 3SAT problem on 5 variables.

Let qm be the probability that m of those clauses, distinct but otherwise selected at random, are satisfiable. Thus qm=Qm/(Nm), where Qm is the number of ways to choose m of the N clauses so that at least one Boolean vector x = x_1 ...xn satisfies them all. Figure 83 illustrates these probabilities when k = 3 and n = 5. Suppose we’re being given distinct random clauses one by one. According to Fig. 83, the chances are better than 77% that we’ll still be able to satisfy them after 20 different clauses have been received, because q_20 ≈ 0.776. But by the time we’ve accumulated 30 of the 80 clauses, the chance of satisfiability has dropped to q_30 ≈ 0.179; and after ten more we reach q_40 ≈ 0.016.

Fig. 83. The probability qm that m distinct clauses of 3SAT on 5 variables are simultaneously satisfiable, for 0 ≤ m ≤ 80.

The illustration makes it appear as if qm = 1 for m < 15, say, and as if qm = 0 for m > 55. But q_8 is actually less than 1, because of (6); exercise 179 gives the exact value. And q_70 is greater than 0, because Q_70 = 32; indeed, every Boolean vector x satisfies exactly (2k−1)(nk)=(1−2−k)N of the N possible k-clauses, so it’s no surprise that 70 noncontradictory 3-clauses on 5 variables can be found. Of course those clauses will hardly ever be the first 70 received, in a random situation. The actual value of q_70 is 32/1646492110120 ≈ 2 × 10−11.

Fig. 84. The total number Tm of different Boolean vectors x = x_1 ... x_5 that simultaneously satisfy m distinct clauses of 3SAT on 5 variables, for 0 ≤ m ≤ 80.

Figure 84 portrays the same process from another standpoint: It shows in how many ways a random set of m clauses can be satisfied. This value, Tm, is a random variable whose mean is indicated in black, surrounded by a gray region that shows the mean plus-or-minus the standard deviation. For example, T_0 is always 32, and T_1 is always 28; but T_2 is either 24, 25, or 26, and it takes these values with the respective probabilities (2200, 480, 480)/3160. Thus the mean for m = 2 is ≈ 24.5, and the standard deviation is ≈ 0.743.

When m = 20, we know from Fig. 83 that T_20 is nonzero more than 77% of the time; yet Fig. 84 shows that T_20 ≈ 1.47 ± 1.17. (Here the notation μ ± σ stands for the mean value μ with standard deviation σ.) It turns out, in fact, that 20 random clauses are uniquely satisfiable, with T_20 = 1, more than 33% of the time; and the probability that T_20 > 4 is only 0.013. With 30 clauses, satisfiability gets dicier and dicier: T_30 ≈ 0.20 ± 0.45; indeed, T_30 is less than 2, more than 98% of the time — although it can be as high as 11 if the clause-provider is being nice to us. By the time 40 clauses are reached, the odds that T_40 exceeds 1 are less than 1 in 4700. Figure 85 shows the probability that Tm = 1 as m varies.

Fig. 85. Pr(Tm = 1), the probability that m distinct clauses of 3SAT on 5 variables are uniquely satisfiable, for 0 ≤ m ≤ 80.

Let P be the number of clauses that have been received when we’re first unable to satisfy them all. Thus we have P = m with probability pm, where pm = qm−1 − qm is the probability that m − 1 random clauses are satisfiable but m are not. These probabilities are illustrated in Fig. 86. Is it surprising that Figs. 85 and 86 look roughly the same? (See exercise 183.)

The expected “stopping time,” E P, is by definition equal to Σm mpm; and it’s not difficult to see, for example by using the technique of summation by parts (exercise 1.2.7–10), that we can compute it by summing the probabilities in Fig. 83:

E  P=Σm qm.(74)

Fig. 86. The stopping time probabilities, pm, that m distinct clauses of 3SAT on 5 variables have just become unsatisfiable, for 0 ≤ m ≤ 80.

The variance of P, namely E(P − E P)2 = (E P^2) − (E P)2, also has a simple expression in terms of the q’s, because

E  P2=Σm (2m+1)qm.(75)

In Figs. 83 and 86 we have E P ≈ 25.22, with variance ≈ 35.73.

So far we’ve been focusing our attention on 3SAT problems, but the same ideas apply also to kSAT for other clause sizes k. Figure 87 shows exact results for the probabilities when n = 5 and 1 ≤ k ≤ 4. Larger values of k give clauses that are easier to satisfy, so they increase the stopping time. With five variables the typical stopping times for random 1SAT, 2SAT, 3SAT, and 4SAT turn out to be respectively 4.06 ± 1.19, 11.60 ± 3.04, 25.22 ± 5.98, and 43.39 ± 7.62. In general if Pk,n is the stopping time for kSAT on n variables, we let

Sk,n=E   Pk,n(76)

be its expected value.

Fig. 87. Extension of Fig. 83 to clauses of other sizes.

Our discussions so far have been limited in another way too: We’ve been assuming that m distinct clauses are being presented to a SAT solver for solution. In practice, however, it’s much easier to generate clauses by allowing repetitions, so that every clause is chosen without any dependence on the past history. In other words, there’s a more natural way to approach random satisfiability, by assuming that Nm possible ordered sequences of clauses are equally likely after m steps, not that we have (Nm) equally likely sets of clauses.

Let q^m be the probability that m random clauses C_1∧ ... ∧Cm are satisfiable, where each Cj is randomly chosen from among the N=2k(nk) possibilities in a kSAT problem on n variables. Figure 88 illustrates these probabilities in the case k = 3, n = 5; notice that we always have q^m≥qm. If N is large while m is small, it’s clear that q^m will be very close to qm, because repeated clauses are unlikely in such a case. Still, we must keep in mind that qN is always zero, while q^m is never zero. Furthermore, the “birthday paradox” discussed in Section 6.4 warns us that repetitions aren’t as rare as we might expect. The deviations of q^m from qm are particularly noticeable in small cases such as the scenario of Fig. 88.

Fig. 88. Random 3SAT on 5 variables when the clauses are sampled with replacement. The probabilities q^m are shown with a black line; the smaller probabilities qm of Fig. 83 are shown in gray.

In any event, there’s a direct way to compute q^m from the probabilities qt and the value of N (see exercise 184):

q^m=Σt=0N{mt}t!qt(Nt)/Nm.(77)

And there are surprisingly simple formulas analogous to (74) and (75) for the stopping time P^, where P^m=q^m−1−q^m, as shown in exercise 186:

E P^=Σm=0N−1NN−mqm;(78)

E P^2=Σm=0N−1NN−mqm(1+2(NN−1+...+NN−m)).(79)

These formulas prove that the expected behavior of P^ is very much like that of P, if qm is small whenever m/N isn’t small. In the case k = 3 and n = 5, the typical stopping times P^=30.58±9.56 are significantly larger than those of P; but we are mostly interested in cases where n is large and where q^m is essentially indistinguishable from qm. In order to indicate plainly that the probability q^m depends on k and n as well as on m, we shall denote it henceforth by Sk(m, n):

Sk(m,n)=Pr(m random clauses of kSAT are satisfiable),(80)

where the m clauses are “sampled with replacement” (they needn’t be distinct). Suitable pseudorandom clauses rand(k, m, n, seed) can easily be generated.

Exact formulas appear to be out of reach when n > 5, but we can make empirical tests. For example, extensive experiments on random 3SAT problems by B. Selman, D. G. Mitchell, and H. J. Levesque [Artificial Intelligence 81 (1996), 17–29] showed a dramatic drop in the chances of satisfiability when the number of clauses exceeds about 4.27n. This “phase transition” becomes much sharper as n grows (see Fig. 89).

Similar behavior occurs for random kSAT, and this phenomenon has spawned an enormous amount of research aimed at evaluating the so-called satisfiability thresholds
αk=limn→∞ Sk,n/n.(81)

Fig. 89. Empirical probability data shows that random 3SAT problems rapidly become unsatisfiable when there are more than α_3n clauses, if n is large enough.

Indeed, we can obtain quite difficult kSAT problems by generating approximately αkn random k-clauses, using empirically observed estimates of αk. If n is large, the running time for random 3SAT with 4.3n clauses is typically orders of magnitude larger than it is when the number of clauses is 4n or 4.6n. (And still tougher problems arise in rare instances when we have, say, 3.9n clauses that happen to be unsatisfiable.)

Strictly speaking, however, nobody has been able to prove that the so-called constants αk actually exist, for all k! The empirical evidence is overwhelming; but rigorous proofs for k = 3 have so far only established the bounds
lim infn→∞ S3,n/n≥3.52;lim supn→∞ S3,n/n≤4.49.(82)

[See M. Hajiaghayi and G. B. Sorkin, arXiv:math/0310193 [math.CO] (2003), 8 pages; A. C. Kaporis, L. M. Kirousis, and E. G. Lalas, Random Struct. & Alg. 28 (2006), 444–480; J. Díaz, L. Kirousis, D. Mitsche, and X. Pérez-Giménez, Theoretical Comp. Sci. 410 (2009), 2920–2934.] A “sharp threshold” result has been established by E. Friedgut [J. American Math. Soc. 12 (1999), 1017–1045, 1053–1054], who proved the existence for k ≥ 2 of functions αk(n) with
limn→∞ Sk(⌊(αk(n)−ϵ)n⌋,n)=1,limn→∞ Sk(⌊(αk(n)+ϵ)n⌋,n)=0,(83)

when ε is any positive number. But those functions might not approach a limit. They might, for example, fluctuate periodically, like the “wobble function” that we encountered in Eq. 5.2.2 (47).

The current best guess for α3, based on heuristics of the “survey propagation” technique to be discussed below, is that α_3 = 4.26675±0.00015 [S. Mertens, M. Mézard, and R. Zecchina, Random Structures & Algorithms 28 (2006), 340–373]. Similarly, it appears reasonable to believe that α_4 ≈ 9.931, α_5 ≈ 21.12, α_6 ≈ 43.37, α_7 ≈ 87.79. The α’s grow as Θ(2k) (see exercise 195); and they are known to be constant when k is sufficiently large [see J. Ding, A. Sly, and N. Sun, STOC 47 (2015), 59.68].

Analysis of random 2SAT

Although nobody knows how to prove that random 3SAT problems almost always become unsatisfiable when the number of clauses reaches ≈ 4.27n, the corresponding question for 2SAT does have a nice answer: The satisfiability threshold α_2 equals 1. For example, when the author first tried 1000 random 2SAT problems with a million variables, 999 of them turned out to be satisfiable when there were 960,000 clauses, while all were unsatisfiable when the number of clauses rose to 1,040,000. Figure 90 shows how this transition becomes sharper as n increases.

Fig. 90. Empirical satisfaction probabilities for 2SAT with approximately n random clauses. (When n = 100, the probability doesn’t become negligible until more than roughly 180 clauses have been generated.)

The fact that S2,n ≈ n was discovered in 1991 by V. Chvátal and B. Reed [FOCS 33 (1992), 620–627], and the same result was obtained independently at about the same time by A. Goerdt and by W. Fernandez de la Vega [see J. Comp. Syst. Sci. 53 (1996), 469–486; Theor. Comp. Sci. 265 (2001), 131–146].

The study of this phenomenon is instructive, because it relies on properties of the digraph that characterizes all instances of 2SAT. Furthermore, the proof below provides an excellent illustration of the “first and second moment principles,” equations MPR–(21) and MPR–(22). Armed with those principles, we’re ready to derive the 2SAT threshold:

Theorem C. Let c be a fixed constant. Then
limn→∞S2(⌊cn⌋,n)={1,if c<1;0,ifc>1.(84)

Proof. Every 2SAT problem corresponds to an implication digraph on the literals, with arcs l¯→l′ and l¯′→l for each clause l ∨ l′. We know from Theorem 7.1.1K that a set of 2SAT clauses is satisfiable if and only if no strong component of its implication digraph contains both x and x¯ for some variable x. That digraph has 2m = 2⌊cn⌋ arcs and 2n vertices. If it were a random digraph, well-known theorems of Karp (which we shall study in Section 7.4.4) would imply that only O(log n) vertices are reachable from any given vertex when c < 1, but that there is a unique “giant strong component” of size Ω(n) when c > 1.

The digraph that arises from random 2SAT isn’t truly random, because its arcs come in pairs, u → v and v¯→u¯. But intuitively we can expect that similar behavior will apply to digraphs that are just halfway random. For example, when the author generated a random 2SAT problem with n = 1000000 and m = .99n, the resulting digraph had only two complementary pairs of strong components with more than one vertex, and their sizes were only 2, 2 and 7, 7; so the clauses were easily satisfiable. Adding another .01n clauses didn’t increase the number of nontrivial strong components, and the problem remained satisfiable. But another experiment with m = n = 1000000 yielded a strong component of size 420, containing 210 variables and their complements; that problem was unsatisfiable.

Based on a similar intuition into the underlying structure, Chvátal and Reed introduced the following “snares and snakes” approach to the proof of Theorem C: Let’s say that an s-chain is any sequence of s strictly distinct literals; thus there are 2sns possible s-chains. Every s-chain C corresponds to clauses

(l¯1∨l2),(l¯2∨l3), . . . , (l¯s−1∨ls)(85)

which in turn correspond to two paths
l1→l2→l3→⋅⋅⋅→ls            and             l¯s→⋅⋅⋅→l¯3→l¯2→l¯1(86)

in the digraph. An s-snare (C; t, u) consists of an s-chain C and two indices t and u, where 1 < t ≤ s and 1 ≤ |u| < s; it specifies the clauses (85) together with

(lt∨l1)                     and                 (l¯s∨lu) if u>0,(l¯s∨l¯−u)  if u<0,(87)

representing l¯t→l1 and either ls → l|u| or ls→l¯| u| . The number of possible s-snares is 2s+1(s − 1)2ns. Their clauses are rarely all present when m is small.

Exercise 200 explains how to use these definitions to prove Theorem C in the case c < 1. First we show that every unsatisfiable 2SAT formula contains all the clauses of at least one snare. Then, if we define the binary random variable

X(C;t,u)=[all  clauses  of (C;t,u)  are  present],(88)

it isn’t difficult to prove that the snares of every s-chain C are unlikely:

E X (C;t,u) ≤ms+1/(2n(n−1))s+1.(89)

Finally, letting X be the sum of X(C; t, u) over all snares, we obtain

E  X=Σ E  X (C;t,u)≤Σs≥02s+1 s(s−1)ns  (m2n(n−1))s+1=2n(mn−1−m)3
by Eq. 1.2.9–(20). This formula actually establishes a stronger form of (84), because it shows that E X is only O(n−1/4) when m = n − n3/4 > cn. Thus

S2(⌊n−n3/4⌋,n)≥Pr(X=0)=1−Pr(X>0)≥1−O(n−1/4)(90)

by the first moment principle.

The other half of Theorem C can be proved by using the concept of a t-snake, which is the special case (C; t, −t) of a (2t − 1)-snare. In other words, given any chain (l_1,...,lt,...,l2t−1), with s = 2t − 1 and lt in the middle, a t-snake generates the clauses (85) together with (lt ∨ l_1) and (l¯s∨l¯t). When t = 5, for example, and (l_1,...,l2t−1) = (x_1,...,x_9), the 2t = 10 clauses are

51,1¯2,2¯3,3¯4,4¯5,5¯6,6¯7,7¯8,8¯9,9¯5¯

and they correspond to 20 arcs that loop around to form a strong component as shown here. We will prove that, when c > 1 in (84), the digraph almost always contains such impediments to satisfiability.

Given a (2t − 1)-chain C, where the parameter t will be chosen later, let

XC=[each clause of (C;t,−t) occurs  exactly  once].(91)

The expected value E XC is clearly f(2t), where
f(r)=mr−(2n(n−1)−r)m−r/(2n(n−1))m(92)

is the probability that r specific clauses occur once each. Notice that
f(r)=(m2n(n−1))r (1+O(r2m)+O(rmn2));(93)

thus the relative error will be O(t^2/n) if m = Θ(n) as n → ∞.

Now let X = ∑ XC, summed over all R = 22t−1n2t−1 possible t-snakes C; thus E X = Rf(2t). We want to show that Pr(X > 0) is very nearly 1, using the second moment principle; so we want to show that the expectation E X^2 = (∑C XC) (∑D XD) = ∑C ∑D E XC XD is small. The key observation is that

E   XCXD=f(4t−r)               if C and D  have exactly  r  clauses  in  common.(94)

Let pr be the probability that a randomly chosen t-snake has exactly r clauses in common with the fixed snake (x_1,...,x2t−1). Then

E X2(E X)2=R2Σr=02t pr  f(4t−r)R2f(2t)2=Σr=02tprf(4t−r)f(2t)2=Σr=02tpr(2n(n−1)m)r(1+O(t2n)).(95)

By studying the interaction of snakes (see exercise 201) one can prove that

(2n)rpr=O(t4/n)+O(t)[r≥t]+O(n)[r=2t],for  1≤r≤2t.(96)

Finally then, as explained in exercise 202, we can choose t = ⌊n1/5⌋ and m = ⌊n + n5/6⌋, to deduce a sharper form of (84) when c > 1:

S2(⌊n+n5/6⌋,n)=O(n−1/30).(97)

(Deep breath.) Theorem C is proved.

Much more precise results have been derived by B. Bollobás, C. Borgs, J. T. Chayes, J. H. Kim, and D. B. Wilson, in Random Structures & Algorithms 18 (2001), 201–256. For example, they showed that

S2(⌊n−n3/4⌋,n)=exp(−Θ(n−1/4));    S2(⌊n+n3/4⌋,n)=exp(−Θ(n1/4)).(98)

Resolution

The backtracking process of Algorithms A, B, D, and L is closely connected to a logical proof procedure called resolution. Starting with a family of clauses called “axioms,” there’s a simple rule by which new clauses can be derived from this given set: Whenever both x ∨ A′ and x¯∨A" are in our repertoire of clauses, we’re allowed to derive the “resolvent” clause A = A′ ∨ A″, denoted by (x∨A′)◊(x¯∨A"). (See exercises 218 and 219.)

A proof by resolution consists of a directed acyclic graph (dag) whose vertices are labeled with clauses in the following way: (i) Every source vertex is labeled with an axiom. (ii) Every other vertex has in-degree 2. (iii) If the predecessors of vertex v are v′ and v″, the label of v is C(v)= C(v′) ◊ C(v″).

When such a dag has a sink vertex labeled A, we call it a “resolution proof of A”; and if A is the empty clause, the dag is also called a “resolution refutation.”

The dag of a proof by resolution can be expanded to a binary tree, by replicating any vertex that has out-degree greater than 1. Such a tree is said to be regular if no path from the root to a leaf uses the same variable twice to form a resolvent. For example, Fig. 91 is a regular resolution tree that refutes Rivest’s unsatisfiable axioms (6). All arcs in this tree are directed upwards.

Fig. 91. One way to derive ∊ by resolving the inconsistent clauses (6).

Notice that Fig. 91 is essentially identical to Fig. 82 on page 217, the backtrack tree by which Algorithm D discovers that the clauses of (6) are unsatisfiable. In fact this similarity is no coincidence: Every backtrack tree that records the behavior of Algorithm D on a set of unsatisfiable clauses corresponds to a regular resolution tree that refutes those axioms, unless Algorithm D makes an unnecessary branch. (An unnecessary branch occurs if the algorithm tries x ← 0 and x ← 1 without using their consequences to discover an unsatisfiable subset of axioms.) Conversely, every regular refutation tree corresponds to a sequence of choices by which a backtrack-based SAT solver could prove unsatisfiability.

The reason behind this correspondence isn’t hard to see. Suppose both values of x need to be tried in order to prove unsatisfiability. When we set x ← 0 in one branch of the backtrack tree, we replace the original clauses F by F | x¯, as in (54). The key point is that we can prove the empty clause by resolution from F | x¯ if and only if we can prove x by resolution from F without resolving on x. (See exercise 224.) Similarly, setting x ← 1 corresponds to changing the clauses from F to F | x.

Consequently, if F is an inconsistent set of clauses that has no short refutation tree, Algorithm D cannot conclude that those clauses are unsatisfiable unless it runs for a long time. Neither can Algorithm L, in spite of enhanced lookahead.

R. Impagliazzo and P. Pudlák [SODA 11 (2000), 128–136] have introduced an appealing Prover–Delayer game, with which it’s relatively easy to demonstrate that certain sets of unsatisfiable clauses require large refutation trees. The Prover names a variable x, and the Delayer responds by saying either x ← 0 or x ← 1 or x ← ∗. In the latter case the Prover gets to decide the value of x; but the Delayer scores one point. The game ends when the current assignments have falsified at least one clause. If the Delayer has a strategy that guarantees a score of at least m points, exercise 226 shows that every refutation tree has at least 2m leaves; hence at least 2m − 1 resolutions must be done, and every backtrack-based solver needs Ω(2m) operations to declare the clauses unsatisfiable.

We can apply their game, for example, to the following interesting clauses:

(x¯j⁢j),for⁢  1⁢≤j≤m;(99)(x¯j⁢j∨x¯j⁢k∨xi⁢k),for⁢  1⁢≤i,j,k≤m;(100)(xj1⁢ ∨xj2∨⋅⋅⋅∨ xj⁢m),for⁢  1⁢ ≤j≤m.(101)

There are m^2 variables xjk, for 1 ≤ j, k ≤ m, which we can regard as the incidence matrix for a binary relation ‘j ≺ k’. With this formulation, (99) says that the relation is irreflexive, and (100) says that it’s transitive; thus, (99) and (100) amount to saying that j ≺ k is a partial ordering. Finally, (101) says that, for every j, there’s a k with j ≺ k. So these clauses state that there’s a partial ordering on {1,...,m} in which no element is maximal; and they can’t all be satisfied.

We can, however, always score m − 1 points if we’re playing Delayer in that game, by using the following strategy suggested by Massimo Lauria: At every step we know an ordered set S of elements, regarded as “small”; initially S = ∅, and we’ll have S = {j_1,...,js} when our score is s. Suppose the Prover queries xjk, and s < m−2. If j = k, we naturally reply that xjk ← 0. Otherwise, if j ∉ S and k ∉ S, we respond xjk ← ∗; then s ← s + 1, and js ← j or k according as the Prover specifies xjk ← 1 or xjk ← 0. Otherwise, if j ∈ S and k ∉ S, we respond xjk ← 1; if j ∉ S and k ∈ S, we respond xjk ← 0. Finally, if j = ja ∈ S and k = jb ∈ S, we respond xjk ← [a < b]. These responses always satisfy (99) and (100). And no clause of (101) becomes false until the Delayer is finally asked a question with s = m − 2. Then the response xjk ← ∗ gains another point. We’ve proved

Theorem R. Every refutation tree for the clauses (99), (100), (101) represents at least 2m−1 − 1 resolution steps.

On the other hand, those clauses do have a refutation dag of size O(m^3). Let Ij and Tijk stand for the irreflexivity and transitivity axioms (99) and (100); and let Mjk = xj1 ∨ ‧‧‧ ∨ xjk, so that (101) is Mjm. Then we have

Mi⁢m◊⁢  Ti⁢m⁢k=Mi(m−1)∨x¯m⁢k,for⁢   1≤i,k<m.(102)

Calling this new clause Mi⁢m⁢k′, we can now derive

Mj(m−1)=((⋅⋅⋅((Mm⁢m◊⁢Mj⁢m1′)◊⁢Mj⁢m2′)◊⋅⋅⋅)◊⁢Mj⁢m(m−1)′)◊⁢Im,
for 1 ≤ j < m. Hence (m − 1)2 +(m − 1)m resolutions have essentially reduced m to m − 1. Eventually we can therefore derive M_11; then M_11 ◊ I_1 = ∊. [This elegant refutation is due to G. Stålmarck, Acta Informatica 33 (1996), 277–280.]

The method we’ve just used to obtain Mj(m−1) from Mmm is, incidentally, a special case of a useful general formula called hyperresolution that is easily proved by induction on r:

(⋅⋅⋅((C0∨x1∨⋅⋅⋅∨xr)◊(C1∨x¯1))◊⋅⋅⋅) ◊ (Cr∨x¯r)                                                                                                                        =C0∨C1∨⋅⋅⋅∨ Cr.(103)

*Lower bounds for general resolution

Let’s change our perspective slightly: Instead of visualizing a proof by resolution as a directed graph, we can think of it as a “straight line” resolution chain, analogous to the addition chains of Section 4.6.3 and the Boolean chains of Section 7.1.2. A resolution chain based on m axioms C_1, ... , Cm appends additional clauses Cm+1, ... , Cm+r, each of which is obtained by resolving two previous clauses of the chain. Formally, we have

Ci=Cj(i)◊Ck(i),for⁢⁢ m+1≤i≤m+r,(104)

where 1 ≤ j(i) < i and 1 ≤ k(i) < i. It’s a refutation chain for C_1, ... , Cm if Cm+r = ∊. The tree in Fig. 91, for example, yields the refutation chain

123¯,234¯,341,41¯2,1¯2¯3,2¯3¯4,3¯4¯1¯,4¯12¯,123,12¯4,1¯24¯,1¯2¯3¯,12,12¯,1¯2¯,1¯2,1,1¯,∈
for the axioms (6); and there are many other ways to refute those axioms, such as

123¯,234¯,341,41¯2,1¯2¯3,2¯3¯4,3¯4¯1¯,4¯12¯,12¯3¯,13¯,14,3¯4¯,24,24¯,2,1¯3,3¯4,14¯,3¯,1,1¯,∈(105)

This chain is quite different from Fig. 91, and perhaps nicer: It has three more steps, but after forming ‘12¯3¯’ it constructs only very short clauses.

We’ll see in a moment that short clauses are crucial if we want short chains. That fact turns out to be important when we try to prove that certain easily understood families of axioms are inherently more difficult than (99), (100), and (101), in the sense that they can’t be refuted with a chain of polynomial size.

Consider, for example, the well known “pigeonhole principle,” which states that m+1 pigeons don’t fit in m pigeon-sized holes. If xjk means that pigeon j occupies hole k, for 0 ≤ j ≤ m and 1 ≤ k ≤ m, the relevant unsatisfiable clauses are

(xj1∨xj⁢2∨⋅⋅⋅∨ xj⁢m),for⁢  0≥j≤m;(106)(x¯i⁢k∨x¯j⁢k),for⁢   0⁢≤i<j≤m⁢   and⁢  1≤k≤m.(107)

(“Every pigeon has a hole, but no hole hosts more than one pigeon.”) These clauses increased the pigeonhole principle’s fame during the 1980s, when Armin Haken [Theoretical Computer Science 39 (1985), 297–308] proved that they have no short refutation chain. His result marked the first time that any set of clauses had been shown to be intractable for resolution in general.

It is absolutely necessary that two people have equally many hairs.
— JEAN APPIER HANZELET, Recreation Mathematicque (1624)

Haken’s original proof was rather complicated. But simpler approaches were eventually found, culminating in a method by E. Ben-Sasson and A. Wigderson [JACM 48 (2001), 149–169], which is based on clause length and applies to many other sets of axioms. If α is any sequence of clauses, let us say that its width, written w(α), is the length of its longest clause or clauses. Furthermore, if α_0 = (C_1,...,Cm), we write w(α_0 ⊢ ∊) for the minimum of w(α) over all refutation chains α = (C_1,...,Cm+r) for α_0, and ∥α_0 ⊢ ∊∥ for the minimum length r of all such chains. The following lemma is the key to proving lower bounds with Ben-Sasson and Wigderson’s strategy:

Lemma B. ∥α_0 ⊢ ∊∥ ≥ e(w(α_0 ⊢ ∊)−1)2/(8n) − 2, for clauses in n ≥ w(α_0)2 variables. Thus there’s exponential growth if we have w(α_0) = O(1) and w(α_0 ⊢ ∊) = Ω(n).

Proof. Let α = (C_1,...,Cm+r) be a refutation of α_0 with r = ∥α_0 ⊢ ∊∥. We will say that a clause is “fat” if its length is W or more, where W ≥ w(α_0) is a parameter to be set later. If α\ α_0 contains f fat clauses, those clauses contain at least Wf literals; hence some literal l appears in at least Wf/(2n) of them.

Now α | l, the chain obtained by replacing each clause Cj by Cj | l, is a refutation of α_0 | l that contains at most ⌊ρf⌋ fat clauses, where ρ = 1 − W/(2n). (The clause Cj |l will be ℘ if l ∈ Cj, thus tautological and effectively absent.)

Suppose f < ρ−b for some integer b. We will prove, by induction on b and secondarily on the total length of all clauses, that there’s a refutation β of α_0 such that w(β) ≤ W + b. This assertion holds when b = 0, since W ≥ w(α_0). If b > 0, there’s a refutation β_0 of α_0 |l with w(β_0) ≤ W + b − 1, when we choose l as above, because ρf < ρ1−b and α | l refutes α_0 | l. Then we can form a resolution chain β_1 that derives l¯ from α_0, by inserting l¯ appropriately into clauses of β_0. And there’s a simple chain β_2 that derives the clauses of α_0 | l¯ from α_0 and l¯. There’s also a refutation β_3 of α_0 | l¯ with w(β_3) ≤ W + b, by induction, because α | l¯ refutes α_0 | l¯. Thus the combination β = {β_1, β_2, β_3} refutes α_0, with
w(β) = max(w(β_0) + 1, w(β_2), w(β_3)) ≤ max(W + b, w(α_0), W + b) = W + b.

Finally, exercise 238 chooses W so that we get the claimed bound.

The pigeon axioms are too wide to be inserted directly into Lemma B. But Ben-Sasson and Wigderson observed that a simplified version of those axioms, involving only clauses of 5SAT, is already intractable.

Notice that we can regard the variable xjk as indicating the presence of an edge between aj and bk in a bipartite graph on the vertices A = {a_0,...,am} and B = {b_1,...,bm}. Condition (106) says that each aj has degree ≥ 1, while condition (107) says that each bk has degree ≤ 1. There is, however, a bipartite graph G_0 on those vertices for which each aj has degree ≤ 5 and such that the following strong “expansion” condition is satisfied:

Every⁢ subset⁢  A′ ⊆  A⁢  with⁢   |A′|≤m/3000⁢   has⁢ |∂A′|⁢  in⁢ G0.(108)

Here ∂A′ denotes the bipartite boundary of A′, namely the set of all bk that have exactly one neighbor in A′.

Given such a graph G_0, whose existence is proved (nonconstructively) in exercise 240, we can formulate a restricted pigeonhole principle, by which the pigeonhole clauses are unsatisfiable if we also require x¯j⁢k whenever aj―bk in G_0.

Let α(G_0) denote the resulting clauses, which are obtained when axioms (106) and (107) are conditioned on all such literals x¯j⁢k. Then w(α(G_0)) ≤ 5, and at most 5m + 5 unspecified variables xjk remain. Lemma B tells us that all refutation chains for α(G_0) have length exp Ω(m) if we can prove that they all have width Ω(m). Haken’s theorem, which asserts that all refutation chains for (106) and (107) also have length exp Ω(m), will follow, because any short refutation would yield a short refutation of α(G_0) after conditioning on the x¯j⁢k.

Thus the following result gives our story a happy ending:

Theorem B. The restricted pigeonhole axioms α(G_0) have refutation width
w(α(G0)⊢∈)≥m/6000.(109)

Proof. We can assign a complexity measure to every clause C by defining
μ(C)=min{|A′|  |  A′ ⊆ A⁢ ⁢    and⁢  α(A′)⊢C}.(110)

Here α(A′) is the set of “pigeon axioms” (106) for aj ∈ A′, together with all of the “hole axioms” (107); and α(A′) ⊢ C means that clause C can be proved by resolution when starting with only those axioms. If C is one of the pigeon axioms, this definition makes μ(C) = 1, because we can let A′ = {aj}. And if C is a hole axiom, clearly μ(C) = 0. The subadditive law
μ(C′ ◊⁢   C")≤μ(C′)+μ(C")(111)

also holds, because a proof of C′ ◊ C″ needs at most the axioms of α(A′) ∪ α(A″) if C′ follows from α(A′) and C″ follows from α(A″).

We can assume that m ≥ 6000. And we must have μ(∊) > m/3000, because of the strong expansion condition (108). (See exercise 241.) Therefore every refutation of α(G_0) must contain a clause C with m/6000 ≤ μ(C) < m/3000; indeed, the first clause Cj with μ(Cj) ≥ m/6000 will satisfy this condition, by (111).

Let A′ be a set of vertices with |A′| = μ(C) and α(A′) ⊢ C. Also let bk be any element of ∂A′, with aj its unique neighbor in A′. Since |A′ \ aj| < μ(C), there must be an assignment of variables that satisfies all axioms of α(A′ \ aj), but falsifies C and the pigeon axiom for j. That assignment puts no two pigeons into the same hole, and it places every pigeon of A′ \ aj.

Now suppose C contains no literal of the form xj′k or x¯j′k, for any aj′ ∈ A. Then we could set xj′k ← 0 for all j′, without falsifying any axiom of α(A′ \ aj); and we could then make the axioms of α({aj}) true by setting xjk ← 1. But that change to the assignment would leave C false, contradicting our assumption that α(A′) ⊢ C. Thus C contains some ±xj′k for each bk ∈ ∂A′; and we must have w(C) ≥ |∂A′| ≥ m/6000.

A similar proof establishes a linear lower bound on the refutation width, hence an exponential lower bound on the refutation length, of almost all random 3SAT instances with n variables and ⌊αn⌋ clauses, for fixed α as n → ∞ (see exercise 243), a theorem of V. Chvátal and E. Szemerédi [JACM 35 (1988), 759–768].

Historical notes: Proofs by resolution, in the more general setting of first-order logic, were introduced by J. A. Robinson in JACM 12 (1965), 23–41. [They’re also equivalent to G. Gentzen’s “cut rule for sequents,” Mathematische Zeitschrift 39 (1935), 176–210, III.1.2 1.] Inspired by Robinson’s paper, Gregory Tseytin developed the first nontrivial techniques to prove lower bounds on the length of resolution proofs, based on unsatisfiable graph axioms that are considered in exercise 245. His lectures of 1966 were published in Volume 8 of the Steklov Mathematical Institute Seminars in Mathematics (1968); see A. O. Slisenko’s English translation, Studies in Constructive Mathematics and Mathematical Logic, part 2 (1970), 115–125.

Tseytin pointed out that there’s a simple way to get around the lower bounds he had proved for his graph-oriented problems, by allowing new kinds of proof steps: Given any set of axioms F, we can introduce a new variable z that doesn’t appear anywhere in F, and add three new clauses G={x⁢z,y⁢z,x¯y¯z¯}; here x and y are arbitrary literals of F. It’s clear that F is satisfiable if and only if F ∪ G is satisfiable, because G essentially says that z = NAND(x, y). Adding new variables in this way is somewhat analogous to using lemmas when proving a theorem, or to introducing a memo cache in a computer program.

His method, which is called extended resolution, can be much faster than pure resolution. For example, it allows the pigeonhole clauses (106) and (107) to be refuted in only O(m^4) steps (see exercise 237). It doesn’t appear to help much with certain other classes of problems such as random 3SAT; but who knows?

SAT solving via resolution

The concept of resolution also suggests alternative ways to solve satisfiability problems. In the first place we can use it to eliminate variables: If F is any set of clauses on n variables, and if x is one of those variables, we can construct a set F′ of clauses on the other n − 1 variables in such a way that F is satisfiable if and only if F′ is satisfiable. The idea is simply to resolve every clause of the form x ∨ A′ with every clause of the form x¯⁢ ∨A", and then to discard those clauses.

For example, consider the following six clauses in four variables:

1234,12¯,1¯2¯3¯,1¯3,23¯,34¯.(112)

We can eliminate the variable x_4 by forming 1234⁢ ◊⁢34¯=123. Then we can eliminate x_3 by resolving 123 and 1¯3 with 1¯2¯3¯ and 23¯:

123⁢◊⁢1¯2¯3¯=℘,     123⁢◊⁢23¯=12,     1¯3⁢◊ 1¯2¯3¯=1¯2¯,       1¯3⁢◊⁢23¯=1¯2.

Now we’re left with {12,12¯,1¯2,1¯2¯}, because the tautology ℘ goes away. Eliminating x_2 gives {1,1¯}, and eliminating x_1 gives {∊}; hence (112) is unsatisfiable.

This method, which was originally proposed for hand calculation by E. W. Samson and R. K. Mueller in 1955, works beautifully on small problems. But why is it valid? There are (at least) two good ways to understand the reason. First, it’s easy to see that F′ is satisfiable whenever F is satisfiable, because C′ ◊ C′ is true whenever C′ and C″ are both true. Conversely, if F′ is satisfied by some setting of the other n − 1 variables, that setting must either satisfy A′ for all clauses of the form x ∨ A′, or else it must satisfy A″ for all clauses of the form x¯∨A". (Otherwise neither A′ nor A″ would be satisfied, for some A′ and some A″, and the clause A′ ∨ A″ in F″ would be false.) Thus at least one of the settings x ← 0 or x ← 1 will satisfy F.

Another good way to understand variable elimination is to notice that it corresponds to the elimination of an existential quantifier (see exercise 248).

Suppose p clauses of F contain x and q clauses contain x¯. Then the elimination of x will give us at most pq new clauses, in the worst case; so F′ will have no more clauses than F did, whenever pq ≤ p + q, namely when (p − 1)(q − 1) ≤ 1. This condition clearly holds whenever p = 0 or q = 0; indeed, we called x a “pure literal” when such cases arose in Algorithm A. The condition also holds whenever p = 1 or q = 1, and even when p = q = 2.

Furthermore we don’t always get pq new clauses. Some of the resolvents might turn out to be tautologous, as above; others might be subsumed by existing clauses. (The clause C is said to subsume another clause C′ if C ⊆ C′ , in the sense that every literal of C appears also in C′ . In such cases we can safely discard C′.) And some of the resolvents might also subsume existing clauses.

Therefore repeated elimination of variables doesn’t always cause the set of clauses to explode. In the worst case, however, it can be quite inefficient.

In January of 1972, Stephen Cook showed his students at the University of Toronto a rather different way to employ resolution in SAT-solving. His elegant procedure, which he called “Method I,” essentially learns new clauses by doing resolution on demand:

Algorithm I (Satisfiability by clause learning). Given m nonempty clauses C_1 ∧‧‧‧∧ Cm on n Boolean variables x_1 ... xn, this algorithm either proves them unsatisfiable or finds strictly distinct literals l_1 ... ln that satisfy them all. In the process, new clauses may be generated by resolution (and m will then increase).

I1. [Initialize.] Set d ← 0.

I2. [Advance.] If d = n, terminate successfully (the literals {l_1,...,ld} satisfy {C_1,...,Cm}). Otherwise set d ← d + 1, and let ld be a literal strictly distinct from l_1, ... , ld−1.

I3. [Find falsified Ci.] If none of C_1, ... , Cm are falsified by {l_1,...,ld}, go back to I2. Otherwise let Ci be a falsified clause.

I4. [Find falsified Cj.] (At this point we have l¯d∈Ci⊆{l¯1,...,l¯d}, but no clause is contained in {l¯1,...,l¯d−1}.) Set ld←l¯d. If none of C_1, ... , Cm are falsified by {l_1,...,ld}, go back to I2. Otherwise let l¯d∈Cj⊆{l¯1,...,l¯d}.

I5. [Resolve.] Set m ← m +1, Cm ← Ci ◊ Cj. Terminate unsuccessfully if Cm is empty. Otherwise set d←max{t⁢ |l¯t∈Cm}, i ← m, and return to I4.

In step I5 the new clause Cm cannot be subsumed by any previous clause Ck for k < m, because Ci⁢ ◊⁢ Cj⊆{l¯1,...,l¯d−1}. Therefore, in particular, no clause is generated twice, and the algorithm must terminate.

This description is intentionally vague when it uses the word “let” in steps I2, I3, and I4: Any available literal ld can be selected in step I2, and any falsified clauses Ci and Cj can be selected in steps I3 and I4, without making the method fail. Thus Algorithm I really represents a family of algorithms, depending on what heuristics are used to make those selections.

For example, Cook proposed the following way (“Method IA”) to select ld in step I2: Choose a literal that occurs most frequently in the set of currently unsatisfied clauses that have the fewest unspecified literals. When applied to the six clauses (112), this rule would set l_1 ← 3 and l_2 ← 2 and l_3 ← 1; then step I3 would find Ci=1¯2¯3¯ false. So step I4 would set l3←1¯ and find Cj=12¯ false, and step I5 would learn C7=2¯3¯. (See exercise 249 for the sequel.)

Cook’s main interest when introducing Algorithm I was to minimize the number of resolution steps; he wasn’t particularly concerned with minimizing the running time. Subsequent experiments by R. A. Reckhow [Ph.D. thesis (Univ. Toronto, 1976), 81–84] showed that, indeed, relatively short resolution refutations are found with this approach. Furthermore, exercise 251 demonstrates that Algorithm I can handle the anti-maximal-element clauses (99)–(101) in polynomial time; thus it trounces the exponential behavior exhibited by all backtrack-based algorithms for this problem (see Theorem R).

On the other hand, Algorithm I does tend to fill memory with a great many new clauses when it is applied to large problems, and there’s no obvious way to deal with those clauses efficiently. Therefore Cook’s method did not appear to be of practical importance, and it remained unpublished for more than forty years.

Conflict driven clause learning

Algorithm I demonstrates the fact that unsuccessful choices of literals can lead us to discover valuable new clauses, thereby increasing our knowledge about the characteristics of a problem. When that idea was rediscovered from another point of view in the 1990s, it proved to be revolutionary: Significant industrial instances of SAT with many thousands or even millions of variables suddenly became feasible for the first time.

The name CDCL solver is often given to these new methods, because they are based on “conflict driven clause learning” rather than on classical backtracking. A CDCL solver shares many concepts with the DPLL algorithms that we’ve already seen; yet it is sufficiently different that we can understand it best by developing the ideas from scratch. Instead of implicitly exploring a search tree such as Fig. 82, a CDCL solver is built on the notion of a trail, which is a sequence L_0L_1 ... LF−1 of strictly distinct literals that do not falsify any clause. We can start with F = 0 (the empty trail). As computation proceeds, our task is to extend the current trail until F = n, thus solving the problem, or to prove that no solution exists, by essentially learning that the empty clause is true.

Suppose there’s a clause c of the form l ∨ ā_1 ∨‧‧‧∨ āk, where a_1 through ak are in the trail but l isn’t. Literals in the trail are tentatively assumed to be true, and c must be satisfied; so we’re forced to make l true. In such cases we therefore append l to the current trail and say that c is its “reason.” (This operation is equivalent to what we called “unit propagation” in previous algorithms; those algorithms effectively removed the literals ā_1, ... ,āk when they became false, thereby leaving l as a “unit” all by itself. But our new viewpoint keeps each clause c intact, and knows all of its literals.) A conflict occurs if the complementary literal l¯ is already in the trail, because l can’t be both true and false; but let’s assume for now that no conflicts arise, so that l can legally be appended by setting LF ← l and F ← F + 1.

If no such forcing clause exists, and if F < n, we choose a new distinct literal in some heuristic way, and we append it to the current trail with a “reason” of Λ. Such literals are called decisions. They partition the trail into a sequence of decision levels, whose boundaries can be indicated by a sequence of indices with 0 = i_0 ≤ i_1 < i_2 < i_3 < ‧‧‧; literal Lt belongs to level d if and only if id ≤ t < id+1. Level 0, at the beginning of the trail, is special: It contains literals that are forced by clauses of length 1, if such clauses exist. Any such literals are unconditionally true. Every other level begins with exactly one decision.

Consider, for example, the problem waerden (3, 3; 9) of (9). The first items placed on the trail might be
tLtlevelreason06¯1Λ(a⁢   decision )19¯2Λ(a decision)232396(rearrangement of the clause 369)34¯3Λ(a decision)453546(rearrangement of the clause 456)583846(⁢ rearrangement of the clause 468)62324677¯37¯5¯3¯(rearrangement of the clause 3¯5¯7¯)82¯32¯5¯8¯(a conflict!)(113)

Three decisions were made, and they started levels at i_1 = 0, i_2 = 1, i_3 = 3. Several clauses have been rearranged; we’ll soon see why. And propagations have led to a conflict, because both 2 and 2¯ have been forced. (We don’t actually consider the final entry L_8 to be part of the trail, because it contradicts L_6.)

If the reason for l includes the literal l¯′, we say “l depends directly on l′.” And if there’s a chain of one or more direct dependencies, from l to l_1 to ‧‧‧ to lk = l′, we say simply that “l depends on l′.” For example, 5 depends directly on 4¯ and 6¯ in (113), and 2¯ depends directly on 5 and 8; hence 2¯ depends on 6¯.

Notice that a literal can depend only on literals that precede it in the trail. Furthermore, every literal l that’s forced at level d > 0 depends directly on some other literal on that same level d; otherwise l would already have been forced at a previous level. Consequently l must necessarily depend on the dth decision.

The reason for reasons is that we need to deal with conflicts. We will see that every conflict allows us to construct a new clause c that must be true whenever the existing clauses are satisfiable, although c itself does not contain any existing clause. Therefore we can “learn” c by adding it to the existing clauses, and we can try again. This learning process can’t go on forever, because only finitely many clauses are possible. Sooner or later we will therefore either find a solution or learn the empty clause. That will be nice, especially if it happens sooner.

A conflict clause c on decision level d has the form l¯∨a¯1∨⋅⋅⋅∨a¯k, where l and all the a’s belong to the trail; furthermore l and at least one ai belong to level d. We can assume that l is rightmost in the trail, of all the literals in c. Hence l cannot be the dth decision; and it has a reason, say l∨a¯1′∨⋅⋅⋅∨a¯k′′. Resolving c with this reason gives the clause c′=a¯1∨⋅⋅⋅∨a¯k∨a¯1′∨⋅⋅⋅∨a¯k′′, which includes at least one literal belonging to level d. If more than one such literal is present, then c′ is itself a conflict clause; we can set c ← c′ and repeat the process. Eventually we are bound to obtain a new clause c′ of the form l¯′∨b¯1∨⋅⋅⋅∨b¯r, where l′ is on level d and where b_1 through br are on lower levels.

Such a c′ is learnable, as desired, because it can’t contain any existing clauses. (Every subclause of c′, including c′ itself, would otherwise have given us something to force at a lower level.) We can now discard levels > d′ of the trail, where d′ is the maximum level of b_1 through br; and — this is the punch line — we can append l¯′ to the end of level d′ , with c′ as its reason. The forcing process now resumes at level d′, as if the learned clause had been present all along.

For example, after the conflict in (113), the initial conflict clause is C=2¯5¯8¯, our shorthand notation for x¯2∨x¯5∨x¯8; and its rightmost complemented literal in the trail is 2, because 5 and 8 came earlier. So we resolve c with 246, the reason for 2, and get c′=45¯68¯. This new clause contains complements of three literals from level 3, namely 4¯, 5, and 8; so it’s still a conflict clause. We resolve it with the reason for 8 and get c′=45¯6. Again c′ is a conflict clause. But the result of resolving this conflict with the reason for 5 is c′ = 46, a clause that is falsified by the literals currently on the trail but has only 4¯ at level 3. Good — we have learned ‘46’: In every solution to waerden (3, 3; 9), either x_4 or x_6 must be true.

Thus the sequel to (113) is
tLtlevelreason06¯1Λ(a decision)14146(the newly learned clause)(114)

and the next step will be to begin a new level 2, because nothing more is forced.

Notice that the former level 2 has gone away. We’ve learned that there was no need to branch on the decision variable x_9, because 6¯ already forces 4. This improvement to the usual backtrack regimen is sometimes called “backjumping,” because we’ve jumped back to a level that can be regarded as the root cause of the conflict that was just discovered.

Exercise 253 explores a possible continuation of (114); dear reader, please jump to it now. Incidentally, the clause ‘46’ that we learned in this example involves the complements of former decisions 4¯ and 6¯; but exercise 255 shows that newly learned clauses might not contain any decision variables whatsoever.

The process of constructing the learned clause from a conflict is not as difficult as it may seem, because there’s an efficient way to perform all of the necessary resolution steps. Suppose, as above, that the initial conflict clause is l¯∨a¯1∨⋅⋅⋅∨a¯k. Then we “stamp” each of the literals ai with a unique number s; and we also insert āi into an auxiliary array, which will eventually hold the literals b¯1,...,b¯r, whenever ai is a literal that received its value on a level d′ with 0 < d′ < d. We stamp l too; and we count how many literals of level d have thereby been stamped. Then we repeatedly go back through the trail until coming to a literal Lt whose stamp equals s. If the counter is bigger than 1 at this point, and if Lt’s reason is Lt∨a¯1′∨⋅⋅⋅∨a¯k′′, we look at each ai′, stamping it and possibly putting it into the b array if it had not already been stamped with s. Eventually the count of unresolved literals will decrease to 1; the learned clause is then L¯t∨b¯1∨⋅⋅⋅∨b¯r.

These new clauses might turn out to be quite large, even when we’re solving a problem whose clauses were rather small to start with. For example, Table 3 gives a glimpse of typical behavior in a medium-size problem. It shows the beginning of the trail generated when a CDCL solver was applied to the 2779 clauses of waerden (3, 10; 97), after about 10,000 clauses had been learned. (Recall that this problem tries to find a binary vector x_1x_2 ... x_97 that has no three equally spaced 0s and no ten equally spaced 1s.) Level 18 in the table has just been launched with the decision L44=57¯; and that decision will trigger the setting of many more literals 15, 49, 61, 68, 77, 78, 87, 96¯, ... , eventually leading to a conflict when trying to set L_67. The conflict clause turns out to have length 22:

Table 3
THE FIRST LEVELS OF A MODERATE-SIZE TRAIL

t

Lt

level

reason

t

Lt

level

reason

t

Lt

level

reason

0

53¯

1

Λ

15

70

11

70 36 53

30

08

15

08 46 27

1

55

2

Λ

16

35

12

Λ

31

65

15

65 46 27

2

44

3

Λ

17

39

13

Λ

32

60

15

60 46 53

3

54

4

Λ

18

37¯

14

Λ

33

50¯

15

∗∗

4

43

5

Λ

19

38

14

38 37 36

34

64

15

64 50 36

5

30

6

Λ

20

47

14

47 37 27

35

22

15

22 50 36

6

34

7

Λ

21

17

14

17 37 27

36

24

15

24 50 37

7

45

8

Λ

22

32

14

32 37 27

37

42

15

42 50 46

8

40

9

Λ

23

69

14

69 37 53

38

48

15

48 50 46

9

27¯

10

Λ

24

21

14

21 37 53

39

73

15

73 50 27

10

79

10

79 53 27

25

46¯

15

Λ

40

04

15

04 50 27

11

01

10

01 27 53

26

28

15

28 46 37

41

63

15

63 50 37

12

36¯

11

Λ

27

41

15

41 46 36

42

33

16

Λ

13

18

11

18 36 27

28

26

15

26 46 36

43

51

17

Λ

14

19

11

19 36 53

29

56

15

56 46 36

44

57¯

18

Λ

(Here ∗∗ denotes the previously learned clause 50 ¯⁢ 26¯⁢ 27 30¯⁢ 32¯⁢ 35¯⁢ 38¯⁢ 40¯⁢ 41¯⁢ 44¯⁢ 45¯⁢ 47¯⁢ 50¯⁢ 55¯⁢ 60¯⁢ 65¯⁢ 70¯.)

53⁢ 27⁢ 36⁢ 70¯⁢ 35¯⁢ 37⁢ 69¯⁢ 21¯⁢ 46⁢ 28¯⁢ 56¯⁢ 65¯⁢ 60¯⁢ 50⁢ 64¯⁢ 24¯⁢ 42¯⁢ 73¯⁢ 63¯⁢ 33¯⁢ 51¯⁢ 57.(115)

(Its literals are shown here in order of the appearance of their complements in the trail.) When we see such a monster clause, we might well question whether we really want to “learn” such an obscure fact!

A closer look, however, reveals that many of the literals in (115) are redundant. For example, 70¯ can safely be deleted, because its reason is ‘70 36 53’; both 36 and 53 already appear in (115), hence (115) ◊ (70 36 53) gets rid of 70¯. Indeed, more than half of the literals in this example are redundant, and (115) can be simplified to the much shorter and more memorable clause

53⁢ 27⁢ 36⁢ 35¯⁢ 37⁢ 46⁢ 50⁢ 33¯⁢ 51¯⁢ 57.(116)

Exercise 257 explains how to discover such simplifications, which turn out to be quite important in practice. For example, the clauses learned while proving waerden (3, 10; 97) unsatisfiable had an average length of 19.9 before simplification, but only 11.2 after; simplification made the algorithm run about 33% faster.

Most of the computation time of a CDCL solver is devoted to unit propagation. Thus we need to know when the value of a literal has been forced by previous assignments, and we hope to know it quickly. The idea of “lazy data structures,” used above in Algorithm D, works nicely for this purpose, in the presence of long clauses, provided that we extend it so that every clause now has two watched literals instead of one. If we know that the first two literals of a clause are not false, then we needn’t look at this clause until one of them becomes false, even though other literals in the clause might be repeatedly veering between transient states of true, false, and undefined. And when a watchee does become false, we’ll try to swap it with a nonfalse partner that can be watched instead. Propagations or conflicts will arise only when all of the remaining literals are false.

Algorithm C below therefore represents clauses with the following data structures: A monolithic array called MEM is assumed to be large enough to hold all of the literals in all of the clauses, interspersed with control information. Each clause c = l_0 ∨ l_1 ∨ ‧‧‧ ∨ lk−1 with k > 1 is represented by its starting position in MEM, with MEM[c + j] = lj for 0 ≤ j < k. Its two watched literals are l_0 and l_1, and its size k is stored in MEM[c − 1]. Unit clauses, for which k = 1, are treated differently; they appear in level 0 of the trail, not in MEM.

A learned clause c can be distinguished from an initial clause because it has a relatively high number, with MINL ≤ c < MAXL. Initially MAXL is set equal to MINL, the smallest cell in MEM that is available for learned clauses; then MAXL grows as new clauses are added to the repertoire. The set of learned clauses is periodically culled, so that the less desirable ones don’t clutter up memory and slow things down. Additional information about a learned clause c is kept in MEM[c − 4] and MEM[c − 5], to help with this recycling process (see below).

Individual literals xk and x¯k, for 1 ≤ k ≤ n, are represented internally by the numbers 2k and 2k + 1 as in (57) above. And each of these 2n literals l has a list pointer Wl, which begins a linked list of the clauses in which l is watched. We have Wl = 0 if there is no such clause; but if Wl = c > 0, the next link in this “watch list” is in MEM[c − 2] if l = l_0, in MEM[c − 3] if l = l_1. [See Armin Biere, Journal on Satisfiability, Boolean Modeling and Comp. 4 (2008), 75–97.]

For example, the first few cells of MEM might contain the following data when we are representing the clauses (9) of waerden (3, 3; 9):

i=01234567891011121314151617...MEM[i]=945324615513468214536810...

(Clause 3 is ‘123’, clause 9 is ‘234’, clause 15 is ‘345’, ... , clause 45 is ‘135’, clause 51 is ‘246’, ... ; the watch lists for literals x_1, x_2, x_3, x_4 begin respectively at W_2 = 3, W_4 = 3, W_6 = 9, W_8 = 15.)

The other major data structures of Algorithm C are focused on variables, not clauses. Each variable xk for 1 ≤ k ≤ n has six current attributes S(k), VAL(k), OVAL(k), TLOC(k), HLOC(k), and ACT(k), which interact as follows: S(k) is the “stamp” that’s used during clause formation. If neither xk nor x¯k appears in the current trail, then VAL(k) = −1, and we say that xk and its two literals are “free.” But if Lt = l is a literal of the trail, belonging to level d, we have

VAL(|l|)=2d+(l&1)⁢   and⁢     TLOC(|l|)=t,where⁢  |l|=l≫1,(117)

and we say that l is “true” and l¯ is “false.” Thus a given literal l is false if and only if VAL(|l|) is nonnegative and VAL(|l|) + l is odd. In most cases a watched literal is not false; but there are exceptions to this rule (see exercise 265). The “reason” for literal l’s current value is kept in variable Rl.

The attributes ACT(k) and HLOC(k) tell the algorithm how to select the next decision variable. Each variable xk has an activity score ACT(k), which heuristically estimates its desirability for branching. All of the free variables, and possibly others, are kept in an array called HEAP, which is arranged so that

ACT(HEAP[j])≤ACT⁢ (HEAP[(j−1)≫1])for⁢ ⁢ 0<j<h(118)

when it contains h elements (see Section 5.2.3). Thus HEAP[0] will always be a free variable of maximum activity, if it is free; so it’s the variable that will be chosen to govern the decision when the trail starts to acquire a new level.

Activity scores help the algorithm to focus on recent conflicts. Suppose, for example, that M = 100 conflicts have been resolved, hence 100 clauses have been learned. Suppose further that xj or x¯j was stamped while resolving the conflicts numbered 3, 47, 95, 99, and 100; but xk or x¯k was stamped during conflicts 41, 87, 94, 95, 96, and 97. We could express their recent activity by computing

ACT(j)=ρ0+ρ1+ρ5+ρ53+ρ97,ACT(k)=ρ3+ρ4+ρ5+ρ6+ρ13+ρ59,
where ρ is a damping factor (say ρ = .95), because 100 − 100 = 0, 100 − 99 = 1, 100 − 95 = 5, ... , 100 − 41 = 59. In this particular case j would be considered to be less active than k unless ρ is less than about .8744.

In order to update the activity scores according to this measure, we would have to do quite a bit of recomputation whenever a new conflict occurs: The new scores would require us to multiply all n of the old scores by ρ, then to increase the activity of every newly stamped variable by 1. But there’s a much better way, namely to compute ρ−M = ρ−100 times the scores shown above:

ACT(j)=ρ−3+ρ−47+ρ−95+ρ−99+ρ−100,ACT(k)=ρ−41+⋅⋅⋅+ρ−96+ρ−97.

These newly scaled scores, suggested by Niklas Eén, give us the same information about the relative activity of each variable; and they’re updated easily, because we need to do only one addition per stamped variable when resolving conflicts.

The only problem is that the new scores can become really huge, because ρ−M can cause floating point overflow after the number M of conflicts becomes large. The remedy is to divide them all by 10^100, say, whenever any variable gets a score that exceeds 10^100. The HEAP needn’t change, since (118) still holds.

During the algorithm the variable DEL holds the current scaling factor ρ−M, divided by 10^100 each time all of the activities have been rescaled.

Finally, the parity of OVAL(k) is used to control the polarity of each new decision in step C6. Algorithm C starts by simply making each OVAL(k) odd, although other initialization schemes are possible. Afterwards it sets OVAL(k) ← VAL(k) whenever xk leaves the trail and becomes free, as recommended by D. Frost and R. Dechter [AAAI Conf. 12 (1994), 301–306] and independently by K. Pipatsrisawat and A. Darwiche [LNCS 4501 (2007), 294–299], because experience has shown that the recently forced polarities tend to remain good. This technique is called “sticking” or “progress saving” or “phase saving.”

Algorithm C is based on the framework of a pioneering CDCL solver called Chaff, and on an early descendant of Chaff called MiniSAT that was developed by N. Eén and N. Sörensson [LNCS 2919 (2004), 502–518].

Algorithm C (Satisfiability by CDCL). Given a set of clauses on n Boolean variables, this algorithm finds a solution L_0L_1 ... Ln−1 if and only if the clauses are satisfiable, meanwhile discovering M new ones that are consequences of the originals. After discovering Mp new clauses, it will purge some of them from its memory and reset Mp; after discovering Mf of them, it will flush part of its trail, reset Mf, and start over. (Details of purging and flushing will be discussed later.)

C1. [Initialize.] Set VAL(k) ← OVAL(k) ← TLOC(k) ←−1, ACT(k) ← S(k) ← 0, R2k ← R2k+1 ← Λ, HLOC(k) ← pk − 1, and HEAP[pk − 1] ← k, for 1 ≤ k ≤ n, where p_1 ... pn is a random permutation of {1,...,n}. Then input the clauses into MEM and the watch lists, as described above. Put the distinct unit clauses into L_0L_1 ... LF−1; but terminate unsuccessfully if there are contradictory clauses (l) and (l¯). Set MINL and MAXL to the first available position in MEM. (See exercise 260.) Set i_0 ← d ← s ← M ← G ← 0, h ← n, DEL ← 1.

C2. [Level complete?] (The trail L_0 ... LF−1 now contains all of the literals that are forced by L_0 ... LG−1.) Go to C5 if G = F.

C3. [Advance G.] Set l ← LG and G ← G + 1. Then do step C4 for all c in the watch list of l¯, unless that step detects a conflict and jumps to C7. If there is no conflict, return to C2. (See exercise 261.)

C4. [Does c force a unit?] Let l_0l_1 ... lk−1 be the literals of clause c, where l1=l¯. (Swap l_0 ↔ l_1 if necessary.) If l_0 is true, do nothing. Otherwise look for a literal lj with 1 < j < k that is not false. If such a literal is found, move c to the watch list of lj. But if l_2, ... , lk−1 are all false, jump to C7 if l_0 is also false. On the other hand if l_0 is free, make it true by setting LF ← l_0, TLOC(|l_0|) ← F , VAL(|l_0|) ← 2d +(l_0 & 1), Rl0 ← c, and F ← F +1.

C5. [New level?] If F = n, terminate successfully. Otherwise if M ≥ Mp, prepare to purge excess clauses (see below). Otherwise if M ≥ Mf, flush literals as explained below and return to C2. Otherwise set d ← d + 1 and id ← F.

C6. [Make a decision.] Set k ← HEAP[0] and delete k from the heap (see exercises 262 and 266). If VAL(k) ≥ 0, repeat this step. Otherwise set l ← 2k + (OVAL(k) & 1), VAL(k) ← 2d + (OVAL(k) & 1), LF ← l, TLOC(|l|) ← F, Rl ← Λ, and F ← F + 1. (At this point F = G + 1.) Go to C3.

C7. [Resolve a conflict.] Terminate unsuccessfully if d = 0. Otherwise use the conflict clause c to construct a new clause l¯′∨b¯1∨⋅⋅⋅∨b¯r as described above. Set ACT(|l|) ← ACT(|l|) + DEL for all literals l stamped during this process; also set d′ to the maximum level occupied by {b_1,..., br} in the trail. (See exercise 263. Increasing ACT(|l|) may also change HEAP.)

C8. [Backjump.] While F > id′+1, do the following: Set F ← F − 1, l ← LF , k ← |l|, OVAL(k) ← VAL(k), VAL(k) ← −1, Rl ← Λ; and if HLOC(|l|) < 0 insert k into HEAP (see exercise 262). Then set G ← F and d ← d′.

C9. [Learn.] If d > 0, set c ← MAXL, store the new clause in MEM at position c, and advance MAXL to the next available position in MEM. (Exercise 263 gives full details.) Set M ← M + 1, LF←l¯′, VAL(|l′|)←2d+(l¯′&1), TLOC(|l′ |) ← F , Rl′ ← c, F ← F + 1, DEL ← DEL/ρ, and return to C3.

The high-level operations on data structures in this algorithm are spelled out in terms of elementary low-level steps in exercises 260–263. Exercises 266–271 discuss simple enhancements that were made in the experiments reported below.

Reality check: Although detailed statistics about the performance of Algorithm C on a wide variety of problems will be presented later, a few examples of typical behavior will help now to clarify how the method actually works in practice. Random choices make the running time of this algorithm more variable than it was in Algorithms A, B, D, or L; sometimes we’re lucky, sometimes we’re not.

In the case of waerden (3, 10; 97), the modest 97-variable-and-2779-clause problem that was considered in Table 3, nine test runs of Algorithm C established unsatisfiability after making between 250 and 300 million memory accesses; the median was 272 Mμ. (This is more than twice as fast as our best previous time, which was obtained with Algorithm L.) The average number of decisions made — namely the number of times LF ← l was done in step C6 — was about 63 thousand; this compares to 1701 “nodes” in Algorithm L, step L3, and 100 million nodes in Algorithms A, B, D. About 53 thousand clauses were learned, having an average size of 11.5 literals (after averaging about 19.9 before simplification).

Fig. 92. It is not possible to color the edges of the flower snark graph Jq with three colors, when q is odd. Algorithm C is able to prove this with amazing speed: Computation times (in megamems) are shown for nine trials at each value of q.

Algorithm C often speeds things up much more dramatically, in fact. For example, Fig. 92 shows how it whips through a sequence of three-coloring problems that are based on “flower snarks.” Exercise 176 defines fsnark (q), an interesting set of 42q + 3 unsatisfiable clauses on 18q variables. The running time of Algorithms A, B, D, and L on fsnark (q) is proportional to 2q, so it’s way off the chart — well over a gigamem already when q = 19. But Algorithm C polishes off the case q = 99 in that same amount of time (thus winning by 24 orders of magnitude)! On the other hand, no satisfactory theoretical explanation for the apparently linear behavior in Fig. 92 is presently known.

Certificates of unsatisfiability. When a SAT solver reports that a given instance is satisfiable, it also produces a set of distinct literals from which we can easily check that every clause is satisfied. But if its report is negative — UNSAT — how confident can we be that such a claim is true? Maybe the implementation contains a subtle error; after all, large and complicated programs are notoriously buggy, and computer hardware isn’t perfect either. A negative answer can therefore leave both programmers and users unsatisfied, as well as the problem.

We’ve seen that unsatisfiability can be proved rigorously by constructing a resolution refutation, namely a chain of resolution steps that ends with the empty clause ∊, as in Fig. 91. But such refutations amount to the construction of a huge directed acyclic graph.

A much more compact characterization of unsatisfiability is possible. Let’s say that the sequence of clauses (C_1, C_2,...,Ct) is a certificate of unsatisfiability for a family of clauses F if Ct = ∊, and if we have

F∧C1∧⋅⋅⋅∧Ci−1∧C¯i⊢1∈for⁢  1≤i≤t.(119)

Here the subscript 1 in ‘G ⊢1 ∊’ means that the clauses G lead to a contradiction by unit propagation; and if Ci is the clause (a_1 ∨ ‧‧‧ ∨ ak), then C¯i is an abbreviation for the conjunction of unit clauses (ā_1) ∧ ‧‧‧ ∧ (āk).

For example, let F = R be Rivest’s clauses (6), which were proved unsatisfiable in Fig. 91. Then (12, 1, 2, ∊) is a certificate of unsatisfiability, because

R∧1¯∧2¯⊢13¯⊢1  4¯⊢1∈(using⁢  123¯,234¯,and⁢ 341)R∧12∧1¯⊢1 2⊢1 4¯⊢13¯⊢1∈(using⁢ 12,4¯12¯, 2¯3¯4, and⁢ 341);R∧12∧1∧2¯⊢1 4⊢13⊢1∈(using⁢ 41¯2, 234¯, and⁢ 3¯4¯1¯);R∧12∧1∧2⊢13⊢14⊢1∈(using⁢  1¯2¯3, 2¯3¯4,and⁢  3¯4¯1¯).

A certificate of unsatisfiability gives a convincing proof, since (119) implies that each Ci must be true whenever F, C_1, ... , Ci−1 are true. And it’s easy to check whether or not G ⊢ 1 ∊, for any given set of clauses G, because everything is forced and no choices are involved. Unit propagation is analogous to water flowing downhill; we can be pretty sure that it has been implemented correctly, even if we don’t trust the CDCL solver that generated the certificate being checked.

E. Goldberg and Y. Novikov [Proceedings of DATE: Design, Automation and Test in Europe 6, 1 (2003), 886–891] have pointed out that CDCL solvers actually produce such certificates as a natural byproduct of their operation:

Theorem G. If Algorithm C terminates unsuccessfully, the sequence (C_1,C_2, ..., Ct) of clauses that it has learned is a certificate of unsatisfiability.

Proof. It suffices to show that, whenever Algorithm C has learned the clause C′=l¯′∨b¯1∨⋅⋅⋅∨b¯r, unit propagation will deduce ∊ if we append the unit clauses (l′) ∧ (b_1) ∧ ‧‧‧ ∧ (br) to the clauses that the algorithm already knows. The key point is that C′ has essentially been obtained by repeated resolution steps,

C′=(...((C⁢◊⁢Rl1)◊⁢Rl2)◊⋅⋅⋅)◊⁢Rls,(120)

where C is the original conflict clause and Rl_1, Rl_2, ... , Rls are the reasons for each literal that was removed while C′ was constructed in step C7. More precisely, we have C = A_0 and Rli = li ∨ Ai, where all literals of A_0 ∪ A_1 ∪‧‧‧∪ As are false (their complements appear in the trail); and
l¯i∈A0∪⋅⋅⋅∪Ai−1,⁢  for⁢ 1≤i≤s;A0∪A1∪⋅⋅⋅∪ As={l¯′,l¯1,...,l¯s,b¯1,...,b¯r}.(121)

Thus the known clauses, plus b_1, ... , br, and l′ , will force ls using clause Rls. And ls−1 will then be forced, using Rls−1. And so on.

Since the unit literals in this proof are propagated in reverse order ls, ls−1, ... , l_1 from the resolution steps in (120), this certificate-checking procedure has become known as “reverse unit propagation” [see A. Van Gelder, Proc. Int. Symp. on Artificial Intelligence and Math. 10 (2008), 9 pages, online as ISAIM2008].

Notice that the proof of Theorem G doesn’t claim that reverse unit propagation will reconstruct the precise reasoning by which Algorithm C learned a clause. Many different downhill paths to ∊, built from ⊢1 steps, usually exist in a typical situation. We merely have shown that every clause learnable from a single conflict does imply the existence of at least one such downhill path.

Many of the clauses learned during a typical run of Algorithm C will be “shots in the dark,” which turn out to have been aimed in unfruitful directions. Thus the certificates in Theorem G will usually be longer than actually necessary to demonstrate unsatisfiability. For example, Algorithm C learns about 53,000 clauses when refuting waerden (3, 10; 97), and about 135,000 when refuting fsnark (99); but fewer than 50,000 of the former, and fewer than 47,000 of the latter, were actually used in subsequent steps. Exercise 284 explains how to shorten a certificate of unsatisfiability while checking its validity.

An unexpected difficulty arises, however: We might spend more time verifying a certificate than we needed to generate it! For example, a certificate for waerden (3, 10; 97) was found in 272 megamems, but the time needed to check it with straightforward unit-propagations was actually 2.2 gigamems. Indeed, this discrepancy becomes significantly worse in larger problems, because a simple program for checking must keep all of the clauses active in its memory. If there are a million active clauses, there are two million literals being watched; hence every change to a literal will require many updates to the data structures.

The solution to this problem is to provide extra hints to the certificate checker. As we are about to see, Algorithm C does not keep all of the learned clauses in its memory; it systematically purges its collection, so that the total number stays reasonable. At such times it can also inform the certificate checker that the purged clauses will no longer be relevant to the proof.

Further improvements also allow annotated certificates to accommodate stronger proof rules, such as Tseytin’s extended resolution and techniques based on generalized autarkies; see N. Wetzler, M. J. H. Heule, and W. A. Hunt, Jr., LNCS 8561 (2014), 422–429.

Whenever a family of clauses has a certificate of unsatisfiability, a variant of Algorithm C will actually find one that isn’t too much longer. (See exercise 386.)

*Purging unhelpful clauses

After thousands of conflicts have occurred, Algorithm C has learned thousands of new clauses. New clauses guide the search by steering us away from unproductive paths; but they also slow down the propagation process, because we have to watch them.

We’ve seen that certificates can usually be shortened; therefore we know that many of the learned clauses will probably never be needed again. For this reason Algorithm C periodically attempts to weed out the ones that appear to be more harmful than helpful, by ranking the clauses that have accumulated.

I consider that a man’s brain originally is like a little empty attic, and you have to stock it with such furniture as you choose. . . . the skilled workman is very careful indeed as to what he takes into his brain-attic. . . . It is a mistake to think that that little room has elastic walls and can distend to any extent. . . . It is of the highest importance, therefore, not to have useless facts elbowing out the useful ones.

— SHERLOCK HOLMES, in A Study in Scarlet (1887)

Algorithm C initiates a special clause-refinement process as soon as it has learned M ≥ Mp clauses and arrived at a reasonably stable state (step C5). Let’s continue our running example, waerden (3, 10; 97), in order to make the issues concrete. If Mp is so huge that no clauses are ever thrown away, a typical run will learn roughly 48 thousand clauses, and do roughly 800 megamems of computation, before proving unsatisfiability. But if Mp = 10000, it will learn roughly 50 thousand clauses, and the computation time will go down to about 500 megamems. In the latter case the total number of learned clauses in memory will rarely exceed 10 thousand.

Indeed, let’s set Mp = 10000 and take a close look at exactly what happened during the author’s first experiments. Algorithm C paused to reconnoiter the situation after having learned 10002 clauses. At that point only 6252 of those 10002 clauses were actually present in memory, however, because of the clause-discarding mechanism discussed in exercise 271. Some clauses had length 2, while the maximum size was 24 and the median was 11; here’s a complete histogram:

2 9 49 126 216 371 542 719 882 1094 661 540 414 269 176 111 35 20 10 3 1 1 1.

Short clauses tend to be more useful, because they reduce more quickly to units.

A learned clause cannot be purged if it is the reason for one of the literals on the trail. In our example, 12 of the 6252 fell into this category; for instance, 30¯ appeared on level 10 of the trail because ′30¯⁢  33¯⁢ 39¯⁢ 41 42¯⁢ 45¯⁢ 46⁢ 48¯⁢ 54¯⁢ 57¯′ had been learned, and we may need to know that clause in a future resolution step.

The purging process will try to remove at least half of the existing learned clauses, so that at most 3126 remain. We aren’t allowed to touch the 12 reason-bound ones; hence we want to forget 3114 of the other 6240. Which of them should we expel?

Among many heuristics that have been tried, the most successful in practice are based on what Gilles Audemard and Laurent Simon have called “literal block distance” [see Proc. Int. Joint Conference on Artificial Intelligence 21 (2009), 399–404]. They observed that each level of the trail can be considered to be a block of more-or-less related variables; hence a long clause might turn out to be more useful than a short clause, if the literals of the long one all lie on just one or two levels while the literals of the short one belong to three or more.

Suppose all the literals of a clause C = l_1 ∨‧‧‧∨ lr appear in the trail, either positively as lj or negatively as l¯j. We can group them by level so that exactly p + q levels are represented, where p of the levels contain at least one positive lj and the other q contain nothing but l¯j's. Then (p, q) is the signature of C with respect to the trail, and p + q is the literal block distance. For example, the very first clause learned from waerden (3, 10; 97) in the author’s test run was

11¯⁢ 16¯⁢ 21¯⁢ 26¯⁢ 36¯⁢ 46¯⁢ 51¯⁢ 61⁢ 66⁢   91;(122)

later, when it was time to rank clauses for purging, the values and trail levels of those literals were specified by VAL(11), VAL(16), ... , VAL(91), which were

20 21 21 21 20 15 16 8 14 20.

Thus 61 was true on level 8 ≫ 1 = 4; 46¯ and 66 were true on level 15 ≫ 1 = 14 ≫ 1 = 7; 51¯ was false on level 8; the others were a mixture of true and false on level 10; hence (122) had p = 3 and q = 1 with respect to the current trail.

If C has signature (p, q) and C′ has signature (p′, q′), where p ≤ p′ and q ≤ q′ and (p, q) ≠ (p′, q′), we can expect that C is more likely than C′ to be useful in future propagations. The same conclusion is plausible also when p + q = p′ + q′ and p < p′ , because C′ won’t force anything until literals from at least p + 1 different levels change sign. These intuitive expectations are borne out by the following detailed data obtained from waerden (3, 10; 97):

(041722305467991717811913953604044386666323246353652138611760522432912983081122200185986775370000831000000001000000)(019152116153072674107825716102074104866121900134037161440006109411000011000000000000000)

The matrix on the left shows how many of the 6240 eligible clauses had a given signature (p, q), for 1 ≤ p ≤ 7 and 0 ≤ q ≤ 8; the matrix on the right shows how many would have been used to resolve future conflicts, if none of them had been removed. There were, for example, 536 learned clauses with p = q = 3, of which only 86 actually turned out to be useful. This data is illustrated graphically in Fig. 93, which shows gray rectangles whose areas correspond to the left matrix, overlaid by black rectangles whose areas correspond to the right matrix. We can’t predict the future, but small (p, q) tends to increase the ratio of black to gray.

Fig. 93. Learned clauses that have p positive and q all-negative levels. The gray ones will never be used again. Unfortunately, there’s no easy way to distinguish gray from black without being clairvoyant.

An alert reader will be wondering, however, how such signatures were found, because we can’t compute them for all clauses until all variables appear in the trail — and that doesn’t happen until all clauses are satisfied! The answer [see A. Goultiaeva and F. Bacchus, LNCS 7317 (2012), 30–43] is that it’s quite possible to carry out a “full run” in which every variable is assigned a value, by making only a slight change to the normal behavior of Algorithm C: Instead of resolving conflicts immediately and backjumping, we can carry on after each conflict until all propagations cease, and we can continue to build the trail in the same way until every variable is present on some level. Conflicts may have occurred on several different levels; but we can safely resolve them later, learning new clauses at that time. Meanwhile, a full trail allows us to compute signatures based on VAL fields. And those VAL fields go into the OVAL fields after backjumping, so the variables in each block will tend to maintain their relationships.

The author’s implementation of Algorithm C assigns an eight-bit value

RANGE(c)←min(⌊16(p+α⁢q),255⌋)(123)

to each clause c; here α is a parameter, 0 ≤ α ≤ 1. We also set RANGE(c) ← 0 if c is the reason for some literal in the trail; RANGE(c) ← 256 if c is satisfied at level 0. If there are mj clauses of range j, and if we want to keep at most T clauses in memory, we find the largest j ≤ 256 such that
sj=m0+m1+⋅⋅⋅+mj−1≤T.(124)

Then we retain all clauses for which RANGE(c) < j, together with T − sj “tie-breakers” that have RANGE(c) = j (unless j = 256). When α has the relatively high value 1516=.9375, this rule essentially preserves as many clauses of small literal block distance as it can; and for constant p + q it favors those with small p.

For example, with α=1516 and the data from Fig. 93, we save clauses that have p = (1, 2, 3, 4, 5) when q ≤ (5, 4, 3, 2, 0), respectively. This gives us s_95 = 12 + 3069 clauses, just 45 shy of our target T = 3126. So we also choose 45 tie-breakers from among the 59 clauses that have RANGE(c) = 95, (p, q)= (5, 1).

Tie-breaking can be done by using a secondary heuristic ACT(c), “clause activity,” which is analogous to the activity score of a variable but it is more easily maintained. If clause c has been used to resolve the conflicts numbered 3, 47, 95, 99, and 100, say, then

ACT(c)=ϱ−3+ϱ−47+ϱ−95+ϱ−99+ϱ−100.(125)

This damping factor ϱ (normally .999) is independent of the factor ρ that is used for variable activities. In the case of Fig. 93, if the 59 clauses with (p, q) = (5, 1) are arranged in order of increasing ACT scores, the gray-and-black pattern is

So if we retain the 45 with highest activity, we pick up 8 of the 10 that turn out to be useful. (Clause activities are imperfect predictors, but they are usually somewhat better than this example implies.)

Exercises 287 and 288 present full details of clause purging in accordance with these ideas. One question remains: After we’ve completed a purge, when should we schedule the next one? Successful results are obtained by having two parameters, Δp and δp. Initially Mp = Δp; then after each purge, we set Δp ← Δp + δp and Mp ← Mp +Δp. For example, if Δp = 10000 and δp = 100, purging will occur after approximately 10000, 20100, 30300, 40600, ... , kΔp+(k2)δp, ... clauses have been learned; and the number of clauses at the beginning of the kth round will be approximately 20000 + 200k = 2Δp + 2kδp. (See exercise 289.)

We’ve based this discussion on waerden (3, 10; 97), which is quite a simple problem. Algorithm C’s gain from clause-purging on larger problems is naturally much more substantial. For example, waerden (3, 13; 160) is only a bit larger than waerden (3, 10; 97). With Δp = 10000 and δp = 100, it finishes in 132 gigamems, after learning 9.5 million clauses and occupying only 503 thousand MEM cells. Without purging, it proves unsatisfiability after learning only 7.1 million clauses, yet at well over ten times the cost: 4307 gigamems, and 102 million cells of MEM.

*Flushing literals and restarting

Algorithm C interrupts itself in step C5 not only to purge clauses but also to “flush literals” that may not have been the best choices for decisions in the trail. The task of solving a tough satisfiability problem is a delicate balancing act: We don’t want to get bogged down in the wrong part of the search space; but we also don’t want to lose the fruits of hard work by “throwing out the baby with the bath water.” A nice compromise has been found by Peter van der Tak, Antonio Ramos, and Marijn Heule [J. Satisfiability, Bool. Modeling and Comp. 7 (2011), 133–138], who devised a useful way to rejuvenate the trail periodically by following trends in the activity scores ACT(k).

Let’s go back to Table 3, to illustrate their method. After learning the clause (116), Algorithm C will update the trail by setting L_44 ← 57 on level 17; that will force L45←66¯, because 39, 42, ... , 63 have all become true; and further positive literals 6, 58, 82, 86, 95, 96 will also join the trail in some order. Step C5 might then intervene to suggest that we should contemplate flushing some or all of the F = 52 literals whose values are currently assigned.

The decision literals 53¯, 55, 44, ... , 51 on levels 1, 2, 3, ... , 17 each were selected because they had the greatest current activity scores when their level began. But activity scores are continually being updated, so the old ones might be considerably out of touch with present realities. For example, we’ve just boosted ACT(53), ACT(27), ACT(36), ACT(70), ... , in the process of learning (116) — see (115). Thus it’s quite possible that several of the first 17 decisions no longer seem wise, because those literals haven’t participated in any recent conflicts.

Let xk be a variable with maximum ACT(k), among all of the variables not in the current trail. It’s easy to find such a k (see exercise 290). Now consider, as a thought experiment, what would happen if we were to jump back all the way to level 0 at this point and start over. Recall that our phase-saving strategy dictates that we would set OVAL(j) ← VAL(j) just before setting VAL(j) ← −1, as the variables become unassigned.

If we now restart at step C6 with d ← 1, all variables whose activity exceeds ACT(k) will receive their former values (although not necessarily in the same order), because the corresponding literals will enter the trail either as decisions or as forced propagations. History will more-or-less repeat itself, because the old assignments did not cause any conflicts, and because phases were saved.

Therefore we might as well avoid most of this back-and-forth unsetting and resetting, by reusing the trail and jumping back only partway, to the first level where the current activity scores significantly change the picture:

Set⁢  d′←0.⁢ While⁢  ACT⁢(|Lid′+1|)≥ACT(k), set⁢   d′←d′+1.Then⁢  if⁢ d′<d,jump⁢ back⁢ to⁢ level⁢ d′.(126)

This is the technique called “literal flushing,” because it removes the literals on levels d′ + 1 through d and leaves the others assigned. It effectively redirects the search into new territory, without being as drastic as a full restart.

In Table 3, for example, ACT(49) might exceed the activity score of every other unassigned variable; and it might also exceed ACT(46), the activity of the decision literal 46¯ on level 15. If the previous 14 decision-oriented activities ACT(53), ACT(55), ... , ACT(37) are all ≥ ACT(49), we would flush all the literals L_25, L_26, ... above level d′ = 14, and commence a new level 15.

Notice that some of the flushed literals other than 46¯ might actually have the largest activities of all. In such cases they will re-insert themselves, before 49 ever enters the scene. Eventually, though, the literal 49 will inaugurate a new level before a new conflict arises. (See exercise 291.)

Experience shows that flushing can indeed be extremely helpful. On the other hand, it can be harmful if it causes us to abandon a fruitful line of attack. When the solver is perking along and learning useful clauses by the dozen, we don’t want to upset the applecart by rocking the boat. Armin Biere has therefore introduced a useful statistic called agility, which tends to be correlated with the desirability of flushing at any given moment. His idea [LNCS 4996 (2008), 28– 33] is beautifully simple: We maintain a 32-bit integer variable called AGILITY, initially zero. Whenever a literal l is placed on the trail in steps C4, C6, or C9, we update the agility by setting

AGILITY←AGILITY−(AGILITY≫13)+(((OVAL|l|−VAL(|l|))&1)≪19).(127)

In other words, the fraction AGILITY/2^32 is essentially multiplied by 1 − δ, then increased by δ if the new polarity of l differs from its previous polarity, where δ = 2−13 ≈ .0001. High agility means that lots of the recent propagations are flipping the values of variables and trying new possibilities; low agility means that the algorithm is basically in a rut, spinning its wheels and getting nowhere.

Table 4
TO FLUSH OR NOT TO FLUSH?

Let a = AGILITY/2^32 when setting Mf ← M +Δf , and let ψ = 1/6, θ = 17/16.

If Δf is

then flush if

If Δf is

then flush if

If Δf is

then flush if

1

a ≤ ψ ≈ .17

32

a ≤ θ^5 ψ ≈ .23

1024

a ≤ θ^10 ψ ≈ .31

2

a ≤ θψ ≈ .18

64

a ≤ θ^6 ψ ≈ .24

2048

a ≤ θ^11 ψ ≈ .32

4

a ≤ θ^2 ψ ≈ .19

128

a ≤ θ^7 ψ ≈ .25

4096

a ≤ θ^12 ψ ≈ .34

8

a ≤ θ^3 ψ ≈ .20

256

a ≤ θ^8 ψ ≈ .27

8192

a ≤ θ^13 ψ ≈ .37

16

a ≤ θ^4 ψ ≈ .21

512

a ≤ θ^9 ψ ≈ .29

16384

a ≤ θ^14 ψ ≈ .39

Armed with the notion of agility, we can finally state what Algorithm C does when step C5 finds M ≥ Mf: First Mf is reset to M + Δf, where Δf is a power of two determined by the “reluctant doubling” sequence 〈1, 1, 2, 1, 1, 2, 4, 1, ...〉; that sequence is discussed below and in exercise 293. Then the agility is compared to a threshold, depending on Δf, according to the schedule in Table 4. (The parameter ψ in that table can be raised or lowered, if you want to increase or decrease the amount of flushing.) If the agility is sufficiently small, xk is found and (126) is performed. Nothing changes if the agility is large or if d′ = d; otherwise (126) has flushed some literals, using the operations of step C8.

Monte Carlo algorithms

Let’s turn now to a completely different way to approach satisfiability problems, based on finding solutions by totally heuristic and randomized methods, often called stochastic local search. We often use such methods in our daily lives, even though there’s no guarantee of success. The simplest satisfiability-oriented technique of this kind was introduced by Jun Gu [see SIGART Bulletin 3, 1 (January 1992), 8–12] and by Christos Papadimitriou [FOCS 32 (1991), 163–169] as a byproduct of more general studies:

“Start with any truth assignment. While there are unsatisfied clauses, pick any one, and flip a random literal in it.”

Some programmers are known to debug their code in a haphazard manner, somewhat like this approach; and we know that such “blind” changes are foolish because they usually introduce new bugs. Yet this idea does have merit when it is applied to satisfiability, so we shall formulate it as an algorithm:

Algorithm P (Satisfiability by random walk). Given m nonempty clauses C_1 ∧ ‧‧‧ ∧ Cm on n Boolean variables x_1 ...xn, this algorithm either finds a solution or terminates unsuccessfully after making N trials.

P1. [Initialize.] Assign random Boolean values to x_1 ...xn. Set j ← 0, s ← 0, and t ← 0. (We know that s clauses are satisfied after having made t flips.)

P2. [Success?] If s = m, terminate successfully with solution x_1 ... xn. Otherwise set j ← (j mod m)+1. If clause Cj is satisfied by x_1 ... xn, set s ← s+1 and repeat this step.

P3. [Done?] If t = N, terminate unsuccessfully.

P4. [Flip one bit.] Let clause Cj be (l_1 ∨ ‧‧‧ ∨ lk). Choose a random index i ∈ {1,...,k}, and change variable |li| so that literal li becomes true. Set s ← 1, t ← t + 1, and return to P2.

Suppose, for example, that we’re given the seven clauses R′ of (7). Thus m = 7, n = 4; and there are two solutions, 01∗1. In this case every nonsolution violates a unique clause; for example, 1100 violates the clause 1¯2¯3, so step P4 is equally likely to change 1100 to 0100, 1000, or 1110, only one of which is closer to a solution. An exact analysis (see exercise 294) shows that Algorithm P will find a solution after making 8.25 flips, on the average. That’s no improvement over a brute-force search through all 2n = 16 possibilities; but a small example like this doesn’t tell us much about what happens when n is large.

Papadimitriou observed that Algorithm P is reasonably effective when it’s applied to 2SAT problems, because each flip has roughly a 50-50 chance of making progress in that case. Several years later, Uwe Schöning [Algorithmica 32 (2002), 615–623] discovered that the algorithm also does surprisingly well on instances of 3SAT, even though the flips when k > 2 in step P4 tend to go “the wrong way”:

Theorem U. If the given clauses are satisfiable, and if each clause has at most three literals, Algorithm P will succeed with probability Ω((3/4)n/n) after making at most n flips.

Proof. By complementing variables, if necessary, we can assume that 0 ... 0 is a solution; under this assumption, every clause has at least one negative literal. Let Xt = x_1 + ‧‧‧ + xn be the number of 1s after t flips have been made. Each flip changes Xt by ±1, and we want to show that there’s a nontrivial chance that xt will become 0. After step P1, the random variable X_0 will be equal to q with probability (qn)/2n.

A clause that contains three negative literals is good news for Algorithm P, because it is violated only when all three variables are 1; a flip will always decrease Xt in such a case. Similarly, a violated clause with two negatives and one positive will invoke a flip that makes progress 2/3 of the time. The worst case occurs only when a problematic clause has only one negative literal. Unfortunately, every clause might belong to this worst case, for all we know.

Instead of studying Xt, which depends on the pattern of clauses, it’s much easier to study another random variable Yt defined as follows: Initially Y_0 = X_0; but Yt+1 = Yt − 1 only when step P4 flips the negative literal that has the smallest subscript; otherwise Yt+1 = Yt + 1. For example, after taking care of a violated clause such as x3∨x¯5∨x¯8, we have Xt+1 = Xt+(+1, −1, −1) but Yt+1 = Yt + (+1, −1, +1) in the three possible cases. Furthermore, if the clause contains fewer than three literals, we penalize Yt+1 even more, by allowing it to be Yt − 1 only with probability 1/3. (After a clause such as x4∨x¯6, for instance, we put Yt+1 = Yt−1 in only 2/3 of the cases when x_6 is flipped; otherwise Yt+1 = Yt+1.)

We clearly have Xt ≤ Yt for all t. Therefore Pr(Xt = 0) ≥ Pr(Yt = 0), after t flips have been made; and we’ve defined things so that it’s quite easy to calculate Pr(Yt = 0), because Yt doesn’t depend on the current clause j:

Pr(Yt+1 = Yt − 1) = 1/3 and Pr(Yt+1 = Yt +1) = 2/3 when Yt > 0.

Indeed, the theory of random walks developed in Section 7.2.1.6 tells us how to count the number of scenarios that begin with Y_0 = q and end with Yt = 0, after Yt has increased p times and decreased p + q times while remaining positive for 0 ≤ t < 2p + q. It is the “ballot number” of Eq. 7.2.1.6–(23),

Cp,p+q−1=q2p+q(2p+qp).(128)

The probability that Y_0 = q and that Yt = 0 for the first time when t = 2p + q is therefore exactly
f(p,q)=12n(nq)q2p+q(2p+qp)(13)p+q(23)p.(129)

Every value of p and q gives a lower bound for the probability that Algorithm P succeeds; and exercise 296 shows that we get the result claimed in Theorem U by choosing p = q ≈ n/3.

Theorem U might seem pointless, because it predicts success only with exponentially small probability when N = n. But if at first we don’t succeed, we can try and try again, by repeating Algorithm P with different random choices. And if we repeat it Kn(4/3)n times, for large enough K, we’re almost certain to find a solution unless the clauses can’t all be satisfied.

In fact, even more is true, because the proof of Theorem U doesn’t exploit the full power of Eq. (129). Exercise 297 carries the analysis further, in a particularly instructive way, and proves a much sharper result:

Corollary W. When Algorithm P is applied K(4/3)n times with N = 2n to a set of satisfiable ternary clauses, its success probability exceeds 1 − e−K/2.

If the clauses C_1 ∧‧‧‧∧ Cm are unsatisfiable, Algorithm P will never demonstrate that fact conclusively. But if we repeat it 100(4/3)n times and get no solution, Corollary W tells us that the chances of satisfiability are incredibly small (less than 10−21). So it’s a safe bet that no solution exists in such a case.

Thus Algorithm P has a surprisingly good chance of finding solutions “with its eyes closed,” while walking at random in the gigantic space of all 2n binary vectors; and we can well imagine that even better results are possible if we devise randomized walking methods that proceed with eyes wide open. Therefore many people have experimented with strategies that try to make intelligent choices about which direction to take at each flip-step. One of the simplest and best of these improvements, popularly known as WalkSAT, was devised by B. Selman, H. A. Kautz, and B. Cohen [Nat. Conf. Artificial Intelligence 12 (1994), 337–343]:

Algorithm W (WalkSAT). Given m nonempty clauses C_1 ∧ ‧‧‧ ∧ Cm on n Boolean variables x_1 ...xn, and a “greed-avoidance” parameter p, this algorithm either finds a solution or terminates unsuccessfully after making N trials. It uses auxiliary arrays c_1 ... cn, f_0 ... fm−1, k_1 ... km, and w_1 ... wm.

W1. [Initialize.] Assign random Boolean values to x_1 ... xn. Also set r ← t ← 0 and c_1 ... cn ← 0 ... 0. Then, for 1 ≤ j ≤ m, set kj to the number of true literals in Cj; and if kj = 0, set fr ← j, wj ← r, and r ← r + 1; or if kj = 1 and the only true literal of Cj is xi or x¯i, set ci ← ci + 1. (Now r is the number of unsatisfied clauses, and the f array lists them. The number Ci is the “cost” or “break count” for variable xi, namely the number of additional clauses that will become false if xi is flipped.)

W2. [Done?] If r = 0, terminate successfully with solution x_1 ... xn. Otherwise, if t = N, terminate unsuccessfully.

W3. [Choose j.] Set j ← fq, where q is uniformly random in {0, 1,...,r − 1}. (In other words, choose an unsatisfied clause Cj at random, considering every such clause to be equally likely; exercise 3.4.1–3 discusses the best way to compute q.) Let clause Cj be (l_1 ∨ ‧‧‧ ∨ lk).

W4. [Choose l.] Let c be the smallest cost among the literals {l_1,...,lk}. If c = 0, orif c ≥ 1 and U ≥ p where U is uniform in [0 .. 1), choose l randomly from among the literals of cost c. (We call this a “greedy” choice, because flipping l will minimize the number of newly false clauses.) Otherwise choose l randomly in {l_1,...,lk}.

W5. [Flip l.] Change the value of variable |l|, and update r, c_1 ... cn, f_0 ... fr−1, k_1 ... km, w_1 ... wm to agree with this new value. (Exercise 302 explains how to implement steps W4 and W5 efficiently, with computer-friendly changes to the data structures.) Set t ← t + 1 and return to W2.

If, for example, we try to satisfy the seven clauses of (7) with Algorithm W, as we did earlier with Algorithm P, the choice x_1x_2x_3x_4 = 0110 violates 2¯3¯4; and c_1c_2c_3c_4 turns out to be 0110 in this situation. So step W4 will choose to flip x_4, and we’ll have the solution 0111. (See exercise 303.)

Notice that step W3 focuses attention on clauses whose variables need to change. Furthermore, a literal that appears in the most unsatisfied clauses is most likely to appear in the chosen clause Cj.

If no cost-free flip is available, step W4 makes nongreedy choices with probability p. This policy keeps the algorithm from getting stuck in an unsatisfiable region from which there’s no greedy exit. Extensive experiments by S. Seitz, M. Alava, and P. Orponen [J. Statistical Mechanics (June 2005), P06006:1–27] indicate that the best choice of p is .57 when large random 3SAT problems are being tackled and when N = ∞. For example, with this setting of p, and with m = 4.2n random 3-literal clauses, Algorithm W works fantastically well: It tends to find solutions after making fewer than 10,000n flips when n = 10^4, and fewer than 2500n flips when 10^5 ≤ n ≤ 10^6.

What about the parameter N? Should we set it equal to 2n (as recommended for 3SAT problems with respect to Algorithm P), or perhaps to n^2 (as recommended for 2SAT in exercise 299), or to 2500n (as just mentioned for 3SAT in Algorithm W), or to something else? When we use an algorithm like WalkSAT, whose behavior can vary wildly depending on random choices and on unknown characteristics of the data, it’s often wise to “cut our losses” and to start afresh with a brand new pattern of random numbers.

Exercise 306 proves that such an algorithm always has an optimum cutoff value N = N∗, which minimizes the expected time to success when the algorithm is restarted after each failure. Sometimes N∗ = ∞ is the best choice, meaning that we should always keep plowing ahead; in other cases N∗ is quite small.

But N∗ exists only in theory, and the theory requires perfect knowledge of the algorithm’s behavior. In practice we usually have little or no information about how N should best be specified. Fortunately there’s still an effective way to proceed, by using the notion of reluctant doubling introduced by M. Luby, A. Sinclair, and D. Zuckerman [Information Proc. Letters 47 (1993), 173–180], who defined the interesting sequence

S1,S2,...=1,1,2,1,1,2,4,1,1,2,1,1,2,4,8,1,1,2,1,1,2,4,1,1,2,...⁢.(130)

The elements of this sequence are all powers of 2. Furthermore we have Sn+1 = 2Sn if the number Sn has already occurred an even number of times, otherwise Sn+1 = 1. A convenient way to generate this sequence is to work with two integers (u, v), and to start with (u_1, v_1) = (1, 1); then

(un+1,vn+1)=(un&−un=vn?(un+1,1):(un,2vn)).(131)

The successive pairs are (1, 1), (2, 1), (2, 2), (3, 1), (4, 1), (4, 2), (4, 4), (5, 1), ... , and we have Sn = vn for all n ≥ 1.

The reluctant doubling strategy is to run Algorithm W repeatedly with N = cS_1, cS_2, cS_3, ... , until success is achieved, where c is some constant. Exercise 308 proves that the expected running time X obtained in this way exceeds the optimum by at most a factor of O(log X). Other sequences besides 〈Sn〉 also have this property, and they’re sometimes better (see exercise 311). The best policy is probably to use 〈cSn〉, where c represents our best guess about the value of N∗; in this way we hedge our bets in case c is too small.

The Local Lemma

The existence of particular combinatorial patterns is often established by using a nonconstructive proof technique called the “probabilistic method,” pioneered by Paul Erdős. If we can show that Pr(X) > 0, in some probability space, then X must be true in at least one case. For example [Bull. Amer. Math. Soc. 53 (1947), 292–294], Erdős famously observed that there is a graph G on n vertices such that neither G nor G¯ contains a k-clique, whenever

(nk)<2k(k−1)/2−1.(132)

For if we consider a random graph G, each of whose (nk) edges is present with probability 1/2, and if U is any particular subset of k vertices in G, the probability that either G | U or G¯⁢ | U is a complete graph is clearly 2/2k(k−1)/2. Hence the probability that this doesn’t happen for any of the (nk) subsets U is at least 1−(nk)21−k(k−1)/2. This probability is positive; so such a graph must exist.

The proof just given does not provide any explicit construction. But it does show that we can find such a graph by making at most 1/(1−(nk)21−k(k−1)/2) random trials, on the average, provided that n and k are small enough that we are able to test all (nk) subgraphs in a reasonable amount of time.

Probability calculations of this kind are often complicated by dependencies between the random events being considered. For example, the presence of a clique in one part of a graph affects the likelihood of many other cliques that share some of the same vertices. But the interdependencies are often highly localized, so that “remote” events are essentially independent of each other. László Lovász introduced an important way to deal with such situations early in the 1970s, and his approach has become known as the “Local Lemma” because it has been used to establish many theorems. First published as a lemma on pages 616–617 of a longer paper [Erdős and Lovász, Infinite and Finite Sets, Colloquia Math. Soc. János Bolyai 10 (1975), 609–627], and subsequently extended to a “lopsided” form [P. Erdős and J. Spencer, Discrete Applied Math. 30 (1991), 151–154], it can be stated as follows:

Lemma L. Let A_1, ... , Am be events in some probability space. Let G be a graph on vertices {1,...,m}, and let (p_1,...,pm) be numbers such that

Pr(Ai|A¯j1∩⋅⋅⋅∩A¯j⁢k)≤pi⁢  whenever⁢  k≥0⁢  and⁢  i⁢ —⁢ j1,...,i⁢ —⁢ jk.(133)

Then Pr(A¯1∩⋅⋅⋅∩A¯m)>0 whenever (p_1,...,pm) lies in a certain set R(G).

In applications we think of the Aj as “bad” events, which are undesirable conditions that interfere with whatever we’re trying to find. The graph G is called a “lopsidependency graph” for our application; this name was coined as an extension of Lovász’s original term “dependency graph,” for which the strict condition ‘= pi’ was assumed in place of ‘≤ pi’ in (133).

The set R(G) of probability bounds for which we can guarantee that all bad events can simultaneously be avoided, given (133), will be discussed further below. If G is the complete graph Km, so that (133) simply states that Pr(Ai) ≤ pi, R(G) is clearly {(p_1,...,pm) | (p_1,...,pm) ≥ (0,..., 0) and p_1 + ‧‧‧ + pm < 1}; this is the smallest possible R(G). At the other extreme, if G is the empty graph Km¯, we get {(p_1,...,pm) | 0 ≤ pj < 1 for 1 ≤ j ≤ m}, the largest possible R(G). Adding an edge to G makes R(G) smaller. Notice that, if (p_1,...,pm) is in R(G) and 0≤pj′≤pj for 1 ≤ j ≤ m, then also (p1′,...,pm′)∈R(G).

Lovász discovered an elegant local condition that suffices to make Lemma L widely applicable [see J. Spencer, Discrete Math. 20 (1977), 69–76]:

Theorem L. The probability vector (p_1,...,pm) is in R(G) when there are numbers 0 ≤ θ_1,...,θm < 1 such that
pi=θi⁢   Πi−j⁢ in⁢G(1−θj).⁢(134)

Proof. Exercise 344(e) proves that Pr(A¯1∩⋅⋅⋅∩A¯m)≥(1−θ1)⁢...⁢(1−θm).

James B. Shearer [Combinatorica 5 (1985), 241–245] went on to determine the exact maximum extent of R(G) for all graphs G, as we’ll see later; and he also established the following important special case:

Theorem J. Suppose every vertex of G has degree ≤ d, where d > 1. Then (p,...,p) ∈ R(G) when p ≤ (d − 1)d−1/dd.

Proof. See the interesting inductive argument in exercise 317.

This condition on p holds whenever p ≤ 1/(ed) (see exercise 319).

Further study led to a big surprise: The Local Lemma proves only that desirable combinatorial patterns exist, although they might be rare. But Robin Moser and Gábor Tardos discovered [JACM 57 (2010), 11:1–11:15] that we can efficiently compute a pattern that avoids all of the bad aj, using an almost unbelievably simple algorithm analogous to WalkSAT!

Algorithm M (Local resampling). Given m events {A_1,...,Am} that depend on n Boolean variables {x_1,...,xn}, this algorithm either finds a vector x_1 ...xn for which none of the events is true, or loops forever. We assume that Aj is a function of the variables {xk | k ∈ Ξj} for some given subset Ξj ⊆ {1,...,n}.

Whenever the algorithm assigns a value to xk, it sets xk ← 1 with probability ξk and xk ← 0 with probability 1 − ξk, where ξk is another given parameter.

M1. [Initialize.] For 1 ≤ k ≤ n, set xk ← [U < ξk], where U is uniform in [0 .. 1).

M2. [Choose j.] Set j to the index of any event such that Aj is true. If no such j exists, terminate successfully, having found a solution x_1 ... xn.

M3. [Resample for Aj.] For each k ∈ Ξj, set xk ← [U <ξk ], where U is uniform in [0 .. 1). Return to M2.

(We have stated Algorithm M in terms of binary variables xk purely for convenience. The same ideas apply when each xk has a discrete probability distribution on any set of values, possibly different for each k.)

To tie this algorithm to the Local Lemma, we assume that event Ai holds with probability ≤ pi whenever the variables it depends on have the given distribution. For example, if ai is the event “x_3 ≠ x_5” then pi must be at least ξ_3(1 − ξ_5)+(1 − ξ_3)ξ_5.

We also assume that there’s a graph G on vertices {1,...,m} such that condition (133) is true, and that i −−− j whenever i ≠ j and Ξi ∩ Ξj ≠ ∅. Then G is a suitable dependency graph for {A_1,...,Am}, because the events Aj1, ... , Ak can’t possibly influence Ai when i⁢ —⁢j1,...,i⁢ —⁢ jk. (Those events share no common variables with ai.) We can also sometimes get by with fewer edges by making G a lopsidependency graph; see exercise 351.

Algorithm M might succeed with any given events, purely by chance. But if the conditions of the Local Lemma are satisfied, success can be guaranteed:

Theorem M. If (133) holds with probabilities that satisfy condition (134) of Theorem L, step M3 is performed for aj at most θj/(1 − θj) times, on average.

Proof. Exercise 352 shows that this result is a corollary of the more general analysis that is carried out below. The stated upper bound is good news, because θj is usually quite small.

Traces and pieces

The best way to understand why Algorithm M is so efficient is to view it algebraically in terms of “traces.” The theory of traces is a beautiful area of mathematics in which amazingly simple proofs of profound results have been discovered. Its basic ideas were first formulated by P. Cartier and D. Foata [Lecture Notes in Math. 85 (1969)], then independently developed from another point of view by R. M. Keller [JACM 20 (1973), 514–537, 696–710] and A. Mazurkiewicz [“Concurrent program schemes and their interpretations,” DAIMI Report PB 78 (Aarhus University, July 1977)]. Significant advances were made by G. X. Viennot [Lecture Notes in Math. 1234 (1985), 321–350], who presented many wide-ranging applications and explained how the theory could readily be visualized in terms of what he called “heaps of pieces.”

Trace theory is the study of algebraic products whose variables are not necessarily commutative. Thus it forms a bridge between the study of strings (in which, for example, acbbaca is quite distinct from baccaab) and the study of ordinary commutative algebra (in which both of those examples are equal to aaabbcc = a^3b^2c^2). Each adjacent pair of letters {a, b} either commutes, meaning that ab = ba, or clashes, meaning that ab is different from ba. If, for instance, we specify that a commutes with c but that b clashes with both a and c, then acbbaca is equal to cabbaac, and it has six variants altogether; similarly, there are ten equally good ways to write baccaab.

Formally speaking, a trace is an equivalence class of strings that can be converted to each other by repeatedly interchanging pairs of adjacent letters that don’t clash. But we don’t need to fuss about the fact that equivalence classes are present; we can simply represent a trace by any one of its equivalent strings, just as we don’t distinguish between equivalent fractions such as 1/2 and 3/6.

Every graph whose vertices represent distinct letters defines a family of traces on those letters, when we stipulate that two letters clash if and only if they are adjacent in the graph. For example, the path graph a −−− b −−− c corresponds to the rules stated above. The distinct traces for this graph are

∈,a,b,c,a⁢a,a⁢b,a⁢c,b⁢a, b⁢b,b⁢c,c⁢b,c⁢c,a⁢a⁢a, a⁢a⁢b,...⁢,c⁢c⁢b,c⁢c⁢c,a⁢a⁢a⁢a,...(135)

if we list them first by size and then in lexicographic order. (Notice that ca is absent, because ac has already appeared.) The complete graph Kn defines traces that are the same as strings, when nothing commutes; the empty graph Kn defines traces that are the same as monomials, when everything commutes. If we use the path a −−− b −−− c −−− d −−− e −−− f to define clashes, the traces bcebafdc and efbcdbca turn out to be the same.

Viennot observed that partial commutativity is actually a familiar concept, if we regard the letters as “pieces” that occupy “territory.” Pieces clash if and only if their territories overlap; pieces commute if and only if their territories are disjoint. A trace corresponds to stacking the pieces on top of one another, from left to right, letting each new piece “fall” until it either rests on the ground or on another piece. In the latter case, it must rest on the most recent piece with which it clashes. He called this configuration an empilement — a nice French word.

More precisely, each piece a is assigned a nonempty subset T(a) of some universe, and we say that a clashes with b if and only if T (a) ∩ T (b) ≠ ∅. For example, the constraints of the graph a −−− b −−− c −−− d −−− e −−− f arise when we let

T (a)= {1, 2}, T (b) = {2, 3}, T (c) = {3, 4}, ..., T (f) = {6, 7};
then the traces bcebafdc and efbcdbca both have

as their empilement. (Readers who have played the game of Tetris® will immediately understand how such diagrams are formed, although the pieces in trace theory differ from those of Tetris because they occupy only a single horizontal level. Furthermore, each type of piece always falls in exactly the same place; and a piece’s territory T (a) might have “holes” — it needn’t be connected.)

Two traces are the same if and only if they have the same empilement. In fact, the diagram implicitly defines a partial ordering on the pieces that appear; and the number of different strings that represent any given trace is the number of ways to sort that ordering topologically (see exercise 324).

Every trace α has a length, denoted by |α|, which is the number of letters in any of its equivalent strings. It also has a height, written h(α), which is the number of levels in its empilement. For example, |bcebafdc| = 8 and h(bcebafdc)= 4.

Arithmetic on traces

To multiply traces, we simply concatenate them. If, for example, α = bcebafdc is the trace corresponding to (136), then ααR = bcebafdccdfabecb has the following empilement:

The algorithm in exercise 327 formulates this procedure precisely. A moment’s thought shows that |αβ| = |α| + |β|, h(αβ) ≤ h(α) + h(β), and h(ααR) = 2h(α).

Traces can also be divided, in the sense that α = (αβ)/β can be determined uniquely when αβ and β are given. All we have to do is remove the pieces of β from the pieces of αβ, one by one, working our way down from the top of the empilements. Similarly, the value of β = α \ (αβ) can be computed from the traces α and αβ. (See exercises 328 and 329.)

Notice that we could rotate diagrams like (136) and (137) by 90 degrees, thereby letting the pieces “fall” to the left instead of downwards. (We’ve used a left-to-right approach for similar purposes in Section 5.3.4, Fig. 50.) Or we could let them fall upwards, or to the right. Different orientations are sometimes more natural, depending on what we’re trying to do.

We can also add and subtract traces, thereby obtaining polynomials in variables that are only partially commutative. Such polynomials can be multiplied in the normal way; for example, (α + β)(γ − δ) = αγ − αδ + βγ − βδ. Indeed, we can even work with infinite sums, at least formally: The generating function for all traces that belong to the graph a−−−b−−− c is

1+a+b+c+a⁢a+a⁢b+a⁢c+b⁢a+b⁢b+b⁢c+c⁢b+c⁢c+a⁢a⁢a+⋅⋅⋅+c⁢c⁢c⁢+a⁢a⁢a⁢a+⋅⋅⋅.(138)

(Compare with (135); we now use 1, not , to stand for the empty string.)

The infinite sum (138) can actually be expressed in closed form: It equals

11−a−b−c⁢+ a⁢c=1+(a+b+c−a⁢c)+(a+b+c−a⁢c)2+⋅⋅⋅,(139)

an identity that is correct not only when the variables are commutative, but also in the algebra of traces, when variables commute only when they don’t clash.

In their original monograph of 1969, Cartier and Foata showed that the sum of all traces with respect to any graph can be expressed in a remarkably simple way that generalizes (139). Let’s define the Möbius function of a trace α with respect to a graph G by the rule
μG(α)={0,if⁢ hG(α)>1;(−1)|α|,otherwise.(140)

(The classical Möbius function μ(n) for integers, defined in exercise 4.5.2–10, is analogous.) Then the Möbius series for G is defined to be

MG=ΣαμG(α)α,(141)

where the sum is over all traces. This sum is a polynomial, when G is finite, because it contains exactly one nonzero term for every independent set of vertices in G; therefore we might call it the Möbius polynomial. For example, when G is the path a−−− b−−− c, we have MG = 1 − a − b − c + ac, the denominator in (139). Cartier and Foata’s generalization of (139) has a remarkably simple proof:

Theorem F. The generating function TG for the sum of all traces, with respect to any graph G, is 1/MG.

Proof. We want to show that MGTG = 1, in the (partially commutative) algebra of traces. This infinite product is ∑α,β μG(α)αβ = ∑γ∑α,β μG(α)γ[γ = αβ].

Hence we want to show that the sum of μG(α), over all ways to factorize γ = αβ as the product of two traces α and β, is zero whenever γ is nonempty.

But that’s easy. We can assume that the letters are ordered in some arbitrary fashion. Let a be the smallest letter in the bottom level of γ’s empilement. We can restrict attention to cases where α consists of independent (commuting) letters (pieces), because μG(α) = 0 otherwise. Now if α = aα′ for some trace α′ , let β′ = aβ; otherwise we must have β = aβ′ for some trace β′ , and we let α′ = aα. In both cases αβ = α′ β′ , (α′ )′ = α, (β′)′ = β, and μG(α)+ μG(α′ ) = 0. So we’ve grouped all possible factorizations of γ into pairs that cancel out in the sum.

The Möbius series for any graph can be computed recursively via the formula

MG=MG\a−a⁢MG\a*,a*={a}∪{b⁢|  a―b},(142)

where a is any letter (vertex) of G, because we have a ∉ I or a ∈ I whenever I is independent. For example, if G is the path a −−− b −−− c −−− d −−− e −−− f, then G \ a∗ = G | {c, d, e, f} is the path c −−− d −−− e −−− f; repeated use of (142) yields

MG⁢= 1−a−b−c−d−e−f+a⁢c+a⁢d+a⁢e+a⁢f                        +b⁢d+b⁢e+b⁢f+c⁢e+c⁢f+d⁢f−a⁢c⁢e−a⁢c⁢f−a⁢d⁢f−b⁢d⁢f(143)

in this case. Since MG is a polynomial, we can indicate its dependence on the variables by writing MG(a, b, c, d, e, f). Notice that MG is always multilinear (that is, linear in each variable); and MG\a (b, c, d, e, f) = MG(0,b,c,d,e,f).

In applications we often want to replace each letter in the polynomial by a single variable, such as z, and write MG(z). The polynomial in (143) then becomes MG(z) = 1−6z +10z^2 − 4z^3; and we can conclude from Theorem F that the number of traces of length n with respect to G is [zn]1/(1−6z+10z2−4z3)=14(2+2)n+2+14(2−2)n+2−2n+1.

Although (142) is a simple recurrence for MG, we can’t conclude that MG is easy to compute when G is a large and complicated graph. Indeed, the degree of MG is the size of a maximum independent set in G; and it’s NP-hard to determine that number! On the other hand, there are many classes of graphs, such as interval graphs and forests, for which MG can be computed in linear time.

If α is any trace, the letters that can occur first in a string that represents it are called the sources of α; these are the pieces on the bottom level of α’s empilement, also called its minimal pieces. Dually, the letters that can occur last are the sinks of α, its maximal pieces. A trace that has only one source is called a cone; in this case all pieces are ultimately supported by a single piece at the bottom. A trace that has only one sink is, similarly, called a pyramid. Viennot proved a nice generalization of Theorem F in his lecture notes:

MG\A/MG⁢   is⁢ the⁢ sum⁢ of⁢ all⁢ traces⁢ whose⁢ sources⁢ are⁢ contained⁢ in⁢ A.(144)

(See exercise 338; Theorem F is the special case where A is the set of all vertices.) In particular, the cones for which a is the only source are generated by

MG\a/MG−1=a⁢  MG\a*/MG.(145)

*Traces and the Local Lemma

Now we’re ready to see why the theory of traces is intimately connected with the Local Lemma. If G is any graph on the vertices {1,...,m}, we say that R(G) is the set of all nonnegative vectors (p_1,...,pm) such that MG(p1′,...,pm1)>0 whenever 0≤pj′≤pj for 1 ≤ j ≤ m. This definition of R(G) is consistent with the implicit definition already given in Lemma L, because of the following characterization found by J. B. Shearer:

Theorem S

Under condition (133) of Lemma L, (p_1,...,pm) ∈ R(G) implies

Pr(A¯1∩⋅⋅⋅∩A¯m)≥MG(p1,...,pm)>0.(146)

Conversely, if (p_1,...,pm) ∉ R(G), there are events B_1, ... , Bm such that

Pr(Bi| B¯j1∩⋅⋅⋅∩B¯j⁢k)=p⁢i  whenever⁢ k≥0⁢   and⁢ i⁢—⁢j1,...⁢,i⁢ —⁢jk,(147) and⁢  Pr(B¯1∩⋅⋅⋅∩B¯m)=0

Proof. When (p_1,...,pm) ∈ R(G), exercise 344 proves that there’s a unique distribution for events B_1, ... , Bm such that they satisfy (147) and also

Pr(∩j∈JA¯j)≥Pr(∩j∈JB¯j)=MG(p1[1∈J],...,pm[m∈J])(148)

for every subset J ⊆ {1,...,m}. In this “extreme” worst-possible distribution, Pr(Bi ∩ Bj) = 0 whenever i−−−j in G. Exercise 345 proves the converse.

Given a probability vector (p_1,...,pm), let

MG* (z)=MG(p1z,...,pmz).(149)

Theorem F tells us that the coefficient of zn in the power series 1/MG*(z) is the sum of all traces of length n for G. Since this coefficient is nonnegative, we know by Pringsheim’s theorem (see exercise 348) that the power series converges for all z < 1 + δ, where 1 + δ is the smallest real root of the polynomial equation MG*(z)=0; this number δ is called the slack of (p_1,...,pm) with respect to G.

It’s easy to see that (p_1,...,pm) ∈ R(G) if and only if the slack is positive. For if δ ≤ 0, the probabilities (p1′,...,pm′) with pj′=(1+δ)pj make MG = 0. But if δ > 0, the power series converges when z = 1. And (since it represents the sum of all traces) it also converges to the positive number 1/MG if any pj is decreased; hence (p_1,...,pm) lies in R(G) by definition. Indeed, this argument shows that, when (p_1,...,pm) ∈ R(G), we can actually increase the probabilities to ((1 + ∊)p_1,..., (1 + ∊)pm), and they will still lie in R(G) whenever ∊ < δ.

Let’s return now to Algorithm M. Suppose the successive bad events Aj that step M3 tries to quench are X_1, X_2, ... , XN , where N is the total number of times step M3 is performed (possibly N = ∞). To prove that Algorithm M is efficient, we shall show that this random variable N has a small expected value, in the probability space of the independent uniform deviates U that appear in steps M1 and M3. The main idea is that X_1X_2 ... XN is essentially a trace for the underlying graph; hence we can consider it as an empilement of pieces.

Some simple and concrete examples will help to develop our intuition; we shall consider two case studies. In both cases there are m = 6 events A, B, C, D, E, F, and there are n = 7 variables x_1 ... x_7. Each variable is a random bit; thus ξ_1 = ‧‧‧ = ξ_7 = 1/2 in the algorithm. Event A depends on x_1x_2, while B depends on x_2x_3, ... , and F depends on x_6x_7. Furthermore, each event occurs with probability 1/4. In Case 1, each event is true when its substring is ‘10’; thus all events are false if and only if x_1 ... x_7 is sorted — that is, x_1 ≤ x_2 ≤ ‧‧‧ ≤ x_7. In Case 2, each event is true when its substring is ‘11’; thus all events are false if and only if x_1 ... x_7 has no two consecutive 1s.

What happens when we apply Algorithm M to those two cases? One possible scenario is that step M3 is applied N = 8 times, with X_1X_2 ... X_8 = BCEBAFDC . The actual changes to the bits x_1 ... x_7 might then be

(Read x_1 ... x_7 from top to bottom in these diagrams, and scan from left to right. Each module means “replace the two bad bits at the left by two random bits at the right.” In examples such as this, any valid solution x_1 ... x_7 can be placed at the far right; all values to the left of the modules are then forced.)

Notice that these diagrams are like the empilement (136), except that they’ve been rotated 90°. We know from (136) that the same diagram applies to the scenario EFBCDBCA as well as to BCEBAFDC, because they’re the same, as traces. Well ..., not quite! In truth, EFBCDBCA doesn’t give exactly the same result as BCEBAFDC in Algorithm M, if we execute that algorithm as presently written. But the results would be identical if we used separate streams of independent random numbers Uk for each variable xk. Thus we can legitimately equate equivalent traces, in the probability space of our random events.

The algorithm runs much faster in practice when it’s applied to Case 1 than when it’s applied to Case 2. How can that be? Both of the diagrams in (150) occur with the same probability, namely (1/2)7(1/4)8, as far as the random numbers are concerned. And every diagram for Case 1 has a corresponding diagram for Case 2; so we can’t distinguish the cases by the number of different diagrams. The real difference comes from the fact that, in Case 1, we never have two events to choose from in step M2, unless they are disjoint and can be handled in either order. In Case 2, by contrast, we are deluged at almost every step with events that need to be snuffed out. Therefore the scenario at the right of (150) is actually quite unlikely; why should the algorithm pick B as the first event to correct, and then C, rather than A? Whatever method is used in step M2, we’ll find that the diagrams for Case 2 will occur less frequently than dictated by the strict probabilities, because of the decreasing likelihood that any particular event will be worked on next, in the presence of competing choices. (See also exercise 353.)

Worst-case upper bounds on the running time of Algorithm M therefore come from situations like Case 1. In general, the empilement BCEBAFDC in (150) will occur in a run of Algorithm M with probability at most bcebafdc, if we write ‘a’ for the probabilistic upper bound for event A that is denoted by ‘pi’ in (133) when A is ai, and if ‘b’, ... , ‘f’ are similar for B, ... , F. The reason is that bcebafdc is clearly the probability that those events are produced by the independent random variables xk set by the algorithm, if the layers of the corresponding empilement are defined by dependencies between the variable sets Ξj. And even if events in the same layer are dependent (by shared variables) yet not lopsidependent (in the sense of exercise 351), such events are positively correlated; so the FKG inequality of exercise MPR–61, which holds for the Bernoulli-distributed variables of Algorithm M, shows that bcebafdc is an upper bound. Furthermore the probability that step M2 actually chooses B, C, E, B, A, F, D, and C to work on is at most 1.

Therefore, when (p_1,...,pm) ∈ R(G), Algorithm M’s running time is maximized when it is applied to events B_1, ..., Bm that have the extreme distribution (148) of exercise 344. And we can actually write down the generating function for the running time with respect to those extreme events: We have

ΣN≥0⁢ Pr(Algorithm⁢ M on B1,..., Bm⁢ does⁢ N⁢ resamplings) zN=MG*(1)MG*(z),(151)

where MG*(z) is defined in (149), because the coefficient of zN in 1/MG*(z) is the sum of the probabilities of all the traces of length N. Theorem F describes the meaning of 1/MG*(1) as a “formal” power series in the variables pi; we proved it without considering whether or not the infinite sum converges when those variables receive numerical values. But when (p_1,...,pm) ∈ R(G), this series is indeed convergent (it even has a positive “slack”).

This reasoning leads to the following theorem of K. Kolipaka and M. Szegedy [STOC 43 (2011), 235–243]:

Theorem K. If (p_1,...,pm) ∈ R(G), Algorithm M resamples Ξj at most

Ej=pjMG\Aj*(p1,...,pm)/MG(p1,...,pm)(152)

times, on the average. In particular, the expected number of iterations of step M3 is at most E_1 + ‧‧‧ + Em ≤ m/δ, where δ is the slack of (p_1,...,pm).

Proof. The extreme distribution B_1, ... , Bm maximizes the number of times Ξj is resampled, and the generating function for that number in the extreme case is

MG(p1,...,pj−1,pj,pj+1,...,pm)MG(p1,...,pj−1,pjz,pj+1,...,pm).(153)

Differentiating with respect to z, then setting z ← 1, gives (152), because the derivative of the denominator is −pjMG\Aj*(p1,...,pm) by (141).

The stated upper bound on E_1 + ‧‧‧ + Em is proved in exercise 355.

*Message passing

Physicists who study statistical mechanics have developed a significantly different way to apply randomization to satisfiability problems, based on their experience with the behavior of large systems of interacting particles. From their perspective, a set of Boolean variables whose values are 0 or 1 is best viewed as an ensemble of particles that have positive or negative “spin”; these particles affect each other and change their spins according to local attractions and repulsions, analogous to laws of magnetism. A satisfiability problem can be formulated as a joint probability distribution on spins for which the states of minimum “energy” are achieved precisely when the spins satisfy as many clauses as possible.

In essence, their approach amounts to considering a bipartite structure in which each variable is connected to one or more clauses, and each clause is connected to one or more variables. We can regard both variables and clauses as active agents, who continually tweet to their neighbors in this social network. A variable might inform its clauses that “I think I should probably be true”; but several of those clauses might reply, “I really wish you were false.” By carefully balancing these messages against each other, such local interactions can propagate and build up more and more knowledge of distant connections, often converging to a state where the whole network is reasonably happy.

A particular message-passing strategy called survey propagation [A. Braun-stein, M. Mézard, and R. Zecchina, Random Structures & Algorithms 27 (2005), 201–226] has proved to be astonishingly good at solving random satisfiability problems in the “hard” region just before the threshold of unsatisfiability.

Let C be a clause and let l be one of its literals. A “survey message” ηC→l is a fraction between 0 and 1 that represents how urgently C wants l to be true. If ηC→l = 1, the truth of l is desperately needed, lest C be false; but if ηC→l = 0, clause C isn’t the least bit worried about the value of variable |l|. Initially we set each ηC→l to a completely random fraction.

We shall consider an extension of the original survey propagation method [see J. Chavas, C. Furtlehner, M. Mézard, and R. Zecchina, J. Statistical Mechanics (November 2005), P11016:1–25; A. Braunstein and R. Zecchina, Physical Review Letters 96 (27 January 2006), 030201:1–4], which introduces additional “reinforcement messages” ηl for each literal l. These new messages, which are initially all zero, represent an external force that acts on l. They help to focus the network activity by reinforcing decisions that have turned out to be fruitful.

Suppose v is a variable that appears in just three clauses: positively in A and B, negatively in C. This variable will respond to its incoming messages ηA→v, ηB→v, ηC→v¯, ηv and ηv¯ by computing two “flexibility coefficients,” πv and πv¯, using the following formulas:

πv=(1−ηv)(1−ηA→v)(1−ηB→v),πv¯=(1−ηv¯)(1−ηC→v¯).

If, for instance, ηv=ηv¯=0 while ηA→v=ηB→v=ηC→v¯=2/3, then πv = 1/9, πv¯=1/3. The π’s are essentially dual to the η’s, because high urgency corresponds to low flexibility and vice versa. The general formula for each literal l is
πl=(1−ηl)    Πl∈C⁢   (1−ηC→l).(154)

Survey propagation uses these coefficients to estimate variable v’s tendency to be either 1 (true), 0 (false), or ∗ (wild), by computing three numbers
p=(1−πv)πv¯πv+πv¯−πvπv¯,q=(1−πv¯)πvπv+πv¯−πvπv¯,r=πvπv¯πv+πv¯−πvπv¯;(155)

then p + q + r = 1, and (p, q, r) is called the “field” of v, representing respectively (truth, falsity, wildness). The field turns out to be (8/11, 2/11, 1/11) in our example above, indicating that v should probably be assigned the value 1. But if ηA→v and ηB→v had been only 1/3 instead of 2/3, the field would have been (5/17, 8/17, 4/17), and we would probably want v = 0 in order to satisfy clause C. Figure 94 shows lines of constant p − q as a function of πv and πv¯; the most decisive cases (|p − q| ≈ 1) occur at the lower right and upper left.

Fig. 94. Lines of constant bias in a variable’s “field.”

If πv=πv¯=0, there’s no flexibility at all: Variable v is being asked to be both true and false. The field is undefined in such cases, and the survey propagation method hopes that this doesn’t happen.

After each literal l has computed its flexibility, the clauses that involve l or l¯ can use πl and πl¯ to refine their survey messages. Suppose, for example, that C is the clause u∨v¯∨w. It will replace the former messages ηC→u, ηC→v¯, ηC→w by
ηC→u/=γv¯→Cγw→C,ηC→v¯′=γu→Cγw→C,ηC→w′=γu→Cγv¯→C,
where each γl→C is a “bias message” received from literal l,
γl→C=(1−πl¯)πl/(1−ηC→l)πl¯+(1−πl¯) πl/(1−ηC→l),(156)

reflecting l’s propensity to be false in clauses other than C. In general we have
ηC→l′=(Πl′∈Cγl′→C)/γl→C.(157)

(Appropriate conventions must be used to avoid division by zero in formulas (156) and (157); see exercise 359.)

New reinforcement messages ηl′ can also be computed periodically, using the formula
ηl′=κ(πl¯∸πl)πl+πl¯−πlπl¯(158)

for each literal l; here x ∸ y denotes max(x − y, 0), and κ is a reinforcement parameter specified by the algorithm. Notice that ηl¯′>0 only if ηl¯′=0.

For example, here are messages that might be passed when we want to satisfy the seven clauses of (7):

l1l2l3ηC→l1ηC→l2ηC→l3γl1→Cγl2→Cγl3→Clπlηl123¯0003/5001101¯2¯31/50003/51/31¯2/51/2234¯1/50001/33/522/51/22¯3¯40003/500310134001/53/51/303¯2/31/31¯3¯4¯000003/542/51/21¯240000004¯10(159)

(Recall that the only solutions to these clauses are 1¯ 2 3 4 and 1¯ 2 3¯ 4.) In this case the reader may verify that the messages of (159) constitute a “fixed point”: The η messages determine the π’s; conversely, we also have ηC→l′=ηC→l for all clauses C and all literals l, if the reinforcement messages ηl remain constant.

Exercise 361 proves that every solution to a satisfiable set of clauses yields a fixed point of the simultaneous equations (154), (156), (157), with the property that ηl = [l is true in the solution].

Experiments with this message-passing strategy have shown, however, that the best results are obtained by using it only for preliminary screening, with the goal of discovering variables whose settings are most critical; we needn’t continue to transmit messages until every clause is fully satisfied. Once we’ve assigned suitable values to the most delicate variables, we’re usually left with a residual problem that can readily be solved by other algorithms such as WalkSAT.

The survey, reinforcement, and bias messages can be exchanged using a wide variety of different protocols. The following procedure incorporates two ideas from an implementation prepared by C. Baldassi in 2012: (1) The reinforcement strength κ begins at zero, but approaches 1 exponentially. (2) Variables are rated 1, 0, or ∗ after each reinforcement, according as max(p, q, r) in their current field is p, q, or r. If every clause then has at least one literal that is true or ∗, message passing will cease even though some surveys might still be fluctuating.

Algorithm S (Survey propagation). Given m nonempty clauses on n variables, this algorithm tries to assign values to most of the variables in such a way that the still-unsatisfied clauses will be relatively easy to satisfy. It maintains arrays πl and ηl of floating point numbers for each literal l, as well as ηC→l for each clause C and each l ∈ C. It has a variety of parameters: ρ (the damping factor for reinforcement), N_0 and N (the minimum and maximum iteration limits), ∊ (the tolerance for convergence), and ψ (the confidence level).

S1. [Initialize.] Set ηl ← πl ← 0 for all literals l, and ηC→l ← U for all clauses C and l ∈ C, where U is uniformly random in [0 .. 1). Also set i ← 0, ϕ ← 1.

S2. [Done?] Terminate unsuccessfully if i ≥ N. If i is even or i < N_0, go to S5.

S3. [Reinforce.] Set ϕ ← ρϕ and κ ← 1 − ϕ. Replace ηl by ηl′ for all literals l, using (158); but terminate unsuccessfully if πl=πl¯=0.

S4. [Test pseudo-satisfiability.] Go to S5 if there is at least one clause whose literals l all appear to be false, in the sense that πl¯<πl and πl¯<12 exercise 358). Otherwise go happily to S8.

S5. [Compute the π’s.] Compute each πl, using (154); see also exercise 359.

S6. [Update the surveys.] Set δ ← 0. For all clauses C and literals l ∈ C, compute ηC→l′ using (157), and set δ←max(δ,|ηC→l′−ηC→l|),ηC→l←ηC→l′.

S8. [Loop on i.] If δ ≥ ∊, set i ← i + 1 and return to S2.

S8. [Reduce the problem.] Assign a value to each variable whose field satisfies |p − q| ≥ ψ. (Exercise 362 has further details.)

Computational experience — otherwise known as trial and error — suggests suitable parameter values. The defaults ρ = .995, N_0 = 5, N = 1000, ∊ = .01, and ψ = .50 seem to provide a decent starting point for problems of modest size. They worked well, for instance, when the author first tried a random 3SAT problem with 42,000 clauses and 10,000 variables: These clauses were pseudo-satisfiable when i = 143 (although δ ≈ .43 was still rather large); then step S8 fixed the values of 8,282 variables with highly biased fields, and unit propagation gave values to 57 variables more. This process needed only about 218 megamems of calculation. The reduced problem had 1526 2-clauses and 196 3-clauses on 1464 variables (because many other variables were no longer needed); 626 steps of WalkSAT polished it off after an additional 42 kilomems. By contrast, when WalkSAT was presented with the original problem (using p = .57), it needed more than 31 million steps to find a solution after 3.4 gigamems of computation.

Similarly, the author’s first experience applying survey propagation to a random 3SAT problem on n = 10^6 variables with m = 4.2n clauses was a smashing success: More than 800,000 variables were eliminated after only 32.8 gigamems of computation, and WalkSAT solved the residual clauses after 8.5 megamems more. By contrast, pure WalkSAT needed 237 gigamems to perform 2.1 billion steps.

A million-variable problem with 4,250,000 clauses proved to be more challenging. These additional 50,000 clauses put the problem well beyond WalkSAT’s capability; and Algorithm S failed too, with its default parameters. However, the settings ρ = .9999 and N_0 = 9 slowed the reinforcement down satisfactorily, and produced some instructive behavior. Consider the matrix

(39883651307123391741133894670250832956495408430433492541205214481050666510849779656386491838973012224815081075106311807110058812701953284135311721711475106315814147891172691347188542541213024203913722043719342156041218393977263516537912603178126455245451991715807120439161682050193381226333203311532505219644155871180288656309441729193996238097310602482618943147071099379245225363740731404263271626561205571573911634832755914035),
which shows the distribution of πv¯ versus πv (see Fig. 94); for example, ‘3988’ at the upper left means that 3988 of the million variables had πv¯ between 0.0 and 0.1 and πv between 0.9 and 1.0. This distribution, which appeared after δ had been reduced to ≈ 0.0098 by 110 iterations, is terrible — very few variables are biased in a meaningful way. Therefore another run was made with ∊ reduced to .001; but that failed to converge after 1000 iterations. Finally, with ∊ = .001 and N = 2000, pseudo-satisfaction occurred at i = 1373, with the nice distribution

(40667819461045979842714687803129816764933822303142128915610001021875118400000017439900000010663620000010381041000000001015550001011011396300100012194911661724161103120162327406839)

(although δ was now ≈ 1!). The biases were now pronounced, yet not entirely reliable; the ψ parameter had to be raised, in order to avoid a contradiction when propagating unit literals in the reduced problem. Finally, with ψ = .99, more than 800,000 variables could be set successfully. A solution was obtained after 210 gigamems (including 21 megamems for WalkSAT to finish the job).

Even better results occur when step S8 is allowed to backtrack, resetting less-biased variables when problems arise. See R. Marino, G. Parisi, and F. RicciTersenghi, Nature Communications 7, 12996 (2016), 1–8.

Success with survey propagation isn’t guaranteed. But hey, when it works, it’s sometimes the only known way to solve a particularly tough problem.

Algorithm S may be viewed as an extension of the “belief propagation” messages used in the study of Bayesian networks [see J. Pearl, Probabilistic Reasoning in Intelligent Systems (1988), Chapter 4]; it essentially goes beyond Boolean logic on {0, 1} to a three-valued logic on {0, 1, ∗}. Analogous message-passing heuristics had actually been considered much earlier by H. A. Bethe and R. E. Peierls [Proc. Royal Society of London A150 (1935), 552–575], and independently by R. G. Gallager [IRE Transactions IT-8 (1962), 21–28]. For further information see M. Mézard and A. Montanari, Information, Physics, and Computation (2009), Chapters 14–22.

*Preprocessing of clauses

A SAT-solving algorithm will often run considerably faster if its input has been transformed into an equivalent but simpler set of clauses. Such transformations and simplifications typically require data structures that would be inappropriate for the main work of a solver, so they are best considered separately.

Of course we can combine a preprocessor and a solver into a single program; and “preprocessing” techniques can be applied again after new clauses have been learned, if we reach a stage where we want to clean up and start afresh. In the latter case the simplifications are called inprocessing. But the basic ideas are most easily explained by assuming that we just want to preprocess a given family of clauses F. Our goal is to produce nicer clauses F′, which are satisfiable if and only if F is satisfiable.

We shall view preprocessing as a sequence of elementary transformations

F=F0→F1→⋅⋅⋅→Fr=F′,160
where each step Fj → Fj+1 “flows downhill” in the sense that it either (i) eliminates a variable without increasing the number of clauses, or (ii) retains all the variables but decreases the number of literals in clauses. Many different downhill transformations are known; and we can try to apply each of the gimmicks in our repertoire, in some order, until none of them lead to any further progress.

Sometimes we’ll actually solve the given problem, by reaching an F′ that is either trivially satisfiable (∅) or trivially unsatisfiable (contains ∊). But we probably won’t be so lucky unless F was pretty easy to start with, because we’re going to consider only downhill transformations that are quite simple.

Before discussing particular transformations, however, let’s think about the endgame: Suppose F has n variables but F′ has n′ < n. After we’ve fed the clauses F′ into a SAT solver and received back a solution, x1′...xn′′, how can we convert it to a full solution x_1 ... xn of the original problem F? Here’s how: For every transformation Fj → Fj+1 that eliminates a variable xk, we shall specify an erp rule (so called because it reverses the effect of preprocessing). An erp rule for elimination is simply an assignment ‘l ← E’, where l is xk or x¯k, and E is a Boolean expression that involves only variables that have not been eliminated. We undo the effect of elimination by assigning to xk the value that makes l true if and only if E is true.

For example, suppose two transformations remove x and y with the erp rules
x¯←y¯∨z,y←1.

To reverse these eliminations, right to left, we would set y true, then x←z¯.

As the preprocessor discovers how to eliminate variables, it can immediately write the corresponding erp rules to a file, so that those rules don’t consume memory space. Afterwards, given a reduced solution x1′...xn′′, a postprocessor can read that file in reverse order and provide the unreduced solution x_1 ... xn.

Transformation 1. Unit conditioning

If a unit clause ‘(l)’ is present, we can replace F by F|l and use the erp rule l ← 1. This elementary simplification will be carried out naturally by most solvers; but it is perhaps even more important in a preprocessor, since it often enables further transformations that the solver would not readily see. Conversely, other transformations in the preprocessor might enable unit conditionings that will continue to ripple down.

One consequence of unit conditioning is that all clauses of F′ will have length two or more, unless F′ is trivially unsatisfiable.

Transformation 2. Subsumption

If every literal in clause C appears also in another clause C′, we can remove C′ . In particular, duplicate clauses will be discarded. No erp rule is needed, because no variable goes away.

Transformation 3. Self-subsumption

If every literal in C except x¯ appears also in another clause C′, where C′ contains x, we can delete x from C′ because C′ \ x = C ◊ C′ . In other words, the fact that C almost subsumes C′ allows us at least to strengthen C′, without actually removing it. Again there’s no erp rule. [Self-subsumption was called “the replacement principle” by J. A. Robinson in JACM 12 (1965), 39.]

Exercise 374 discusses data structures and algorithms by which subsumptions and self-subsumptions can be discovered with reasonable efficiency.

Transformation 4. Downhill resolution

Suppose x appears only in clauses C_1, ... , Cp and x¯ appears only in C1′,...,Cq′. We’ve observed (see (112)) that variable x can be eliminated if we replace those p + q clauses by the pq clauses {Ci◊Cj′|1≤i≤p,1≤j≤q}. The corresponding erp rule (see exercise 367) is
either x¯←Λi=1p(Ci\x)or x←Λj=1q(Cj′\x¯).(161)

Every variable can be eliminated in this way, but we might be flooded with too many clauses. We can prevent this by limiting ourselves to “downhill” cases, in which the new clauses don’t outnumber the old ones. The condition pq ≤ p+q is equivalent to (p − 1)(q − 1) ≤ 1, as noted above following (112); the variable is always removed in such cases. But the number of new clauses might be small even when pq is large, because of tautologies or subsumption. Furthermore, N. Eén and A. Biere wrote a fundamental paper on preprocessing [LNCS 3569 (2005), 61–75], which introduced important special cases that allow many of the pq potential clauses to be omitted; see exercise 369. Therefore a preprocessor typically tries to eliminate via resolution whenever min(p, q) ≤ 10, say, and abandons the attempt only when more than p+q resolvents have been generated.

Many other transformations are possible, although the four listed above have proved to be the most effective in practice. We could, for instance, look for failed literals: If unit propagation leads to a contradiction when we assume that some literal l is true (namely when F ∧ (l) ⊢1 ∊), then we’re allowed to assume that l is false (because the unit clause (l¯) is certifiable). This observation and several others related to it were exploited in the lookahead mechanisms of Algorithm Y above. But Algorithm C generally has no trouble finding failed literals all by itself, as a natural byproduct of its mechanism for resolving conflicts. Exercises 378–384 discuss other techniques that have been proposed for preprocessing.

Sometimes preprocessing turns out to be dramatically successful. For example, the anti-maximal-element clauses of exercise 228 can be proved unsatisfiable via transformations 1–4 after only about 400 megamems of work when m = 50. Yet Algorithm C spends 3 gigamems on that untransformed problem when m is only 14; and it needs 11 Gμ when m = 15, ... , failing utterly before m = 20.

A more typical example arises in connection with Fig. 78 above: The problem of showing that there’s no 4-step path to involves 8725 variables, 33769 clauses, and 84041 literals, and Algorithm C requires about 6 gigamems to demonstrate that those clauses are unsatisfiable. Preprocessing needs less than 10 megamems to reduce that problem to just 3263 variables, 19778 clauses, and 56552 literals; then Algorithm C can handle those with 5 Gμ of further work.

On the other hand, preprocessing might take too long, or it might produce clauses that are more difficult to deal with than the originals. It’s totally useless on the waerden or langford problems. (Further examples are discussed below.)

Encoding constraints into clauses

Some problems, like waerden (j, k; n), are inherently Boolean, and they’re essentially given to us as native-born ANDs of ORs. But in most cases we can represent a combinatorial problem via clauses in many different ways, not immediately obvious, and the particular encoding that we choose can have an enormous effect on the speed with which a SAT solver is able to crank out an answer. Thus the art of problem encoding turns out to be just as important as the art of devising algorithms for satisfiability.

Our study of SAT instances has already introduced us to dozens of interesting encodings; and new applications often lead to further ideas, because Boolean algebra is so versatile. Each problem may seem at first to need its own special tricks. But we’ll see that several general principles are available for guidance.

In the first place, different solvers tend to like different encodings: An encoding that’s good for one algorithm might be bad for another.

Consider, for example, the at-most-one constraint, y_1 + ‧‧‧ + yp ≤ 1, which arises in a great many applications. The obvious way to enforce this condition is to assert (p2) binary clauses (y¯i∨y¯j), for 1 ≤ i < j ≤ p, so that yi = yj = 1 is but those clauses become unwieldy when p is large. The alternative encoding in exercise 12, due to Marijn Heule, does the same job with only 3p − 6 binary constraints when p ≥ 3, by introducing a few auxiliary variables a_1, ... , a⌊(p−3)/2⌋. When we formulated Langford’s problem in terms of clauses, via (12), (13), and (14) above, we therefore considered two variants called langford (n) and langford′ (n), where the former uses the obvious encoding of at-most-one constraints and the latter uses Heule’s method. Furthermore, exercise 7.1.1–55(b) encoded at-most-one constraints in yet another way, having the same number of binary clauses but about twice as many auxiliary variables; let’s give the name langford″(n) to the clauses that we get from that scheme.

We weren’t ready to discuss which of the encodings works better in practice, when we introduced langford (n) and langford″ (n) above, because we hadn’t yet examined any SAT-solving algorithms. But now we’re ready to reveal the answer; and the answer is: “It depends.” Sometimes langford′ (n) wins over langford (n); sometimes it loses. It always seems to beat langford″(n). Here, for example, are typical statistics, with runtimes rounded to megamems (Mμ) or kilomems (Kμ):

variables clauses Algorithm D Algorithm L Algorithm C

langford (9)

104

1722

23 Mμ

16 Mμ

15 Mμ

(UNSAT)

langford′ (9)

213

801

82 Mμ

16 Mμ

21 Mμ

(UNSAT)

langford″ (9)

335

801

139 Mμ

20 Mμ

24 Mμ

(UNSAT)

langford (13)

228

5875

71685 Mμ

45744 Mμ

295571 Mμ

(UNSAT)

langford′ (13)

502

1857

492992 Mμ

38589 Mμ

677815 Mμ

(UNSAT)

langford″ (13)

795

1857

950719 Mμ

46398 Mμ

792757 Mμ

(UNSAT)

langford (16)

352

11494

5 Mμ

52 Mμ

301 Kμ

(SAT)

langford′ (16)

796

2928

12 Mμ

31 Mμ

418 Kμ

(SAT)

langford″ (16)

1264

2928

20 Mμ

38 Mμ

510 Kμ

(SAT)

langford (64)

6016

869650

(huge)

(bigger)

35 Mμ

(SAT)

langford′ (64)

14704

53184

(huger)

(big)

73 Mμ

(SAT)

langford″ (64)

23488

53184

(hugest)

(biggest)

304 Mμ

(SAT)

Algorithm D prefers langford to langford′, because it doesn’t perform unit propagations very efficiently. Algorithm L, which excels at unit propagation, likes langford′ better. Algorithm C also excels at unit propagation, but it exhibits peculiar behavior: It prefers langford, and on satisfiable instances it zooms in quickly to find a solution; but for some reason it runs very slowly on unsatisfiable instances when n ≥ 10.

Another general principle is that short encodings — encodings with few variables and/or few clauses — are not necessarily better than longer encodings. For example, we often need to use Boolean variables to encode the value of a variable x that actually ranges over d > 2 different values, say 0 ≤ x < d. In such cases it’s natural to use the binary representation x = (xl−1 ... x_0)2, where l = ⌈lg d⌉, and to construct clauses based on the independent bits xj; but that representation, known as the log encoding, surprisingly turns out to be a bad idea in many cases unless d is large. A direct encoding with d binary variables x_0, x_1, ... , xd−1, where xj = [x = j], is often much better. And the order encoding with d − 1 binary variables x^1, ... , xd−1, where xj = [x ≥ j], is often better yet; this encoding was introduced in 1994 by J. M. Crawford and A. B. Baker [AAAI Conf. 12 (1994), 1092–1097]. In fact, exercise 408 presents an important application where the order encoding is the method of choice even when d is 1000 or more! The order encoding is exponentially larger than the log encoding, yet it wins in this application because it allows the SAT solver to deduce consequences rapidly via unit propagation.

Graph coloring problems illustrate this principle nicely. When we tried early in this section to color a graph with d colors, we encoded the color of each vertex with a direct representation, (15); but we could have used binary notation for those colors. And we could also have used the order encoding, even though the numerical ordering of colors is irrelevant in the problem itself. With a log encoding, exercise 391 exhibits three distinct ways to enforce the constraint that adjacent vertices have different colors. With the order encoding, exercise 395 explains that it’s easy to handle graph coloring. And there also are four ways to work with the direct encoding, namely (a) to insist on one color per vertex by including the at-most-one exclusion clauses (17); or (b) to allow multivalued (multicolored) vertices by omitting those clauses; or (c) to actually welcome multicolored vertices, by omitting (17) and forcing each color class to be a kernel, as suggested in answer 14; or (d) to include (17) but to replace the “preclusion” clauses (16) by so-called “support” clauses as explained in exercise 399.

These eight options can be compared empirically by trying to arrange 64 colored queens on a chessboard so that no queens of the same color appear in the same row, column, or diagonal. That task is possible with 9 colors, but not with 8. By symmetry we can prespecify the colors of all queens in the top row.

encoding

colors

variables

clauses

Algorithm L

Algorithm C

univalued

8

512

7688

3333 Mμ

9813 Mμ

(UNSAT)

multivalued

8

512

5896

1330 Mμ

11997 Mμ

(UNSAT)

kernel

8

512

6408

4196 Mμ

12601 Mμ

(UNSAT)

support

8

512

13512

16796 Mμ

20990 Mμ

(UNSAT)

log(a)

8

2376

5120

(immense)

20577 Mμ

(UNSAT)

log(b)

8

192

5848

(enormous)

15033 Mμ

(UNSAT)

log(c)

8

192

5848

(enormous)

15033 Mμ

(UNSAT)

order

8

448

6215

43615 Mμ

5122 Mμ

(UNSAT)

univalued

9

576

8928

2907 Mμ

464 Mμ

(SAT)

multivalued

9

576

6624

104 Mμ

401 Mμ

(SAT)

kernel

9

576

7200

93 Mμ

87 Mμ

(SAT)

support

9

576

15480

2103 Mμ

613 Mμ

(SAT)

log(a)

9

3168

6776

(gigantic)

1761 Mμ

(SAT)

log(b)

9

256

6776

(colossal)

1107 Mμ

(SAT)

log(c)

9

256

6584

(mammoth)

555 Mμ

(SAT)

order

9

512

7008

(monstrous)

213 Mμ

(SAT)

(Each running time shown here is the median of nine runs, made with different random seeds.) It’s clear from this data that the log encodings are completely unsuitable for Algorithm L; and even the order encoding confuses that algorithm’s heuristics. But Algorithm L shines over Algorithm C with respect to most of the direct encodings. On the other hand, Algorithm C loves the order encoding, especially in the difficult unsatisfiable case.

And that’s not the end of the story. H. Tajima [M.S. thesis, Kobe University (2008)] and N. Tamura noticed that order encoding has another property, which beats all other encodings with respect to graph coloring: Every k-clique of vertices {v_1,...,vk} in a graph allows us to append two additional “hint clauses”

(v¯1d−k+1∨⋅⋅⋅∨v¯kd−k+1)∧(v1k−1∨⋅⋅⋅∨vkk−1)(162)

to the clauses for d-coloring — because some vertex of the clique must have a color ≤ d − k, and some vertex must have a color ≥ k − 1. With these additional clauses, the running time to prove unsatisfiability of the 8-coloring problem drops drastically to just 60 Mμ with Algorithm L, and to only 13 Mμ with Algorithm C. We can even reduce it to just 2 Mμ(!) by using that idea twice (see exercise 396).

The order encoding has several other nice properties, so it deserves a closer look. When we represent a value x in the range 0 ≤ x < d by the binary variables xj = [x ≥ j] for 1 ≤ j < d, we always have
x=x1+x2+⋅⋅⋅+xd−1;(163)

hence order encoding is often known as unary representation. The axiom clauses

(x¯j+1∨xj)for 1≤j<d−1(164)

are always included, representing the fact that x ≥ j +1 implies x ≥ j for each j; these clauses force all the 1s to the left and all the 0s to the right. When d = 2 the unary representation reduces to a one-bit encoding equal to x itself; when d = 3 it’s a two-bit encoding with 00, 10, and 11 representing 0, 1, and 2.

We might not know all of the bits xj of x’s unary encoding while a problem is in the course of being solved. But if we do know that, say, x^3 = 1 and x^7 = 0, then we know that x belongs to the interval [3 .. 7).

Suppose we know the unary representation of x. Then no calculation is necessary if we want to know the unary representation of y = x + a, when a is a constant, because yj = xj−a. Similarly, z = a − x is equivalent to zj=x¯a+1−j; and w = ⌊x/a⌋ is equivalent to wj = xaj. Out-of-bounds superscripts are easy to handle in formulas such as this, because xi = 1 when i ≤ 0 and xi = 0 when i ≥ d. The special case x¯=d−1−x is obtained by left-right reflection of x¯1...⁢ x¯j−1:

(d−1−x)j=(x¯)j=xd−j¯.(165)

If we are using the order encoding for two independent variables x and y, with 0 ≤ x, y < d, it’s similarly easy to encode the additional relation x ≤ y + a:

x−y≤a⁢ ⇔  x≤y+a⁢ ⇔  Λj=max(0,a+1)min(d−1,d+a)(x¯j∨yj−a).(166)

And there are analogous ways to place bounds on the sum, x + y:

x+y≤a⇔x≤y¯+a+1−d⇔Λj=max(0,a+2−d)min(d−1,a+1)(x¯j∨y¯a+1−j);(167)

x+y≥a⁢ ⇔  x¯≤y−a−1+d⁢ ⇔  Λj=max(1,a+1−d)min(d,a)(xj∨ya+1−j).(168)

In fact, exercise 405 shows that the general condition ax + by ≤ c can be enforced with at most d binary clauses, when a, b, and c are constant. Any set of such relations, involving at most two variables per constraint, is therefore a 2SAT problem.

Relations between three or more order-encoded variables can also be handled without difficulty, as long as d isn’t too large. For example, conditions such as x + y ≤ z and x + y ≥ z can be expressed with O(d log d) clauses of length ≤ 3 (see exercise 407). Arbitrary linear inequalities can also be represented, in principle. But of course we shouldn’t expect SAT solvers to compete with algebraic methods on problems that are inherently numerical.

Another constraint of great importance in the encoding of combinatorial problems is the relation of lexicographic order: Given two bit vectors x_1 ...xn and y_1 ...yn, we want to encode the condition (x_1 ...xn)2 ≤ (y_1 ... yn)2 as a conjunction of clauses. Fortunately there’s a nice way to do this with just 3n − 2 ternary clauses involving n − 1 auxiliary variables a_1, ... , an−1, namely

Λk=1n−1((x¯k∨yk∨a¯k−1)∧(x¯k∨ak∨a¯k−1)∧(yk∨ak∨a¯k−1))∧(x¯n∨yn∨a¯n−1),(169)

where ‘ā_0’ is omitted. For example, the clauses

(x¯1∨y1)∧(x¯1∨a1)∧(y1∨a1)∧(x¯2∨y2∨a¯1)∧(x¯2∨a2∨a¯1)∧(y2∨a2∨a¯1)∧(x¯3∨y3∨a¯2)

assert that x_1x_2x_3 ≤ y_1y_2y_3. And the same formula, but with the final term (x¯n∨yn∨a¯n−1) replaced by (x¯n∨a¯n−1)∧(yn∨a¯n−1), works for the strict comparison x_1 ... xn < y_1 ... yn. These formulas arise by considering the carries that occur when (x¯1⁢ ...⁢x¯n)2+(1⁢ or 0) is added to (y_1 ... yn)2. (See exercise 415.)

The general problem of encoding a constraint on the Boolean variables x_1, ... , xn is the question of finding a family of clauses F that are satisfiable if and only if f(x_1,...,xn) is true, where f is a given Boolean function. We usually introduce auxiliary variables a_1, ... , am into the clauses of F, unless f can be expressed directly with a short CNF formula; thus the encoding problem is to find a “good” family F such that we have
f(x1,...,xn)=1⁢   ⇔ ∃a1⁢ ...⁢ ∃am⁢   ΛC∈FC,(170)

where each C is a clause on the variables {a_1,...,am, x_1,...,xn}. The variables a_1, ... , am can be eliminated by resolution as in (112), at least in principle, leaving us with a CNF for f — although that CNF might be huge. (See exercise 248.)

If there’s a simple circuit that computes f, we know from (24) and exercise 42 that there’s an equally simple “Tseytin encoding” F, with one auxiliary variable for each gate in the circuit. For example, suppose we want to encode the condition x_1 ... xn ≠ y_1 ... yn. The shortest CNF expression for this function f(x_1,...,xn, y_1,...,yn) has 2n clauses (see exercise 413); but there’s a simple circuit (Boolean chain) with just n + 1 gates:

a1←x1⁢ ⊕ y1,…,an←xn⁢ ⊕ yn,f←a1 ∨…∨ an.

Using (24) we get the 4n clauses

∧j=1n((x¯j∨yj∨aj)∧(xj∨y¯j∨aj)∧(xj∨yj∨a¯j)∧(x¯j∨y¯j∨a¯j)),(171)

together with (a_1 ∨ … ∨ an), as a representation of ‘x_1 ... xn ≠ y_1 ... yn’.

But this is overkill; D. A. Plaisted and S. Greenbaum have pointed out [Journal of Symbolic Computation 2 (1986), 293 304] that we can often avoid about half of the clauses in such situations. Indeed, only 2n of the clauses (171) are necessary (and sufficient), namely the ones involving a¯j:

∧j=1n((xj∨yj∨a¯j)∧(x¯j∨y¯j∨a¯j)).(172)

The other clauses are “blocked” (see exercise 378) and unhelpful. Thus it’s a good idea to examine whether all of the clauses in a Tseytin encoding are really needed. Exercise 416 illustrates another interesting case.

An efficient encoding is possible also when f has a small BDD, and in general whenever f can be computed by a short branching program. Recall the example “Pi function” introduced in 7.1.1 (22); we observed in 7.1.2 (6) that it can be written (((x2∧x¯4)⊕x¯3)∧x¯1)⊕x2. Thus it has a 12-clause Tseytin encoding

(x2∨a¯1)∧(x¯4∨a¯1)∧(x¯2∨x4∨a1)∧(x3∨a1∨a2)∧(x¯3∨a¯1∨a2)∧(x¯3∨a1∨a¯2)∧(x3∨a¯1∨a¯2)∧(x¯1∨a¯3)∧(a2∨a¯3)∧(x1∨a¯2∨a3)∧(x2∨a3)∧(x¯2∨a¯3).

The Pi function also has a short branching program, 7.1.4 (8), namely

I8=(1¯?7:6),I7=(2¯?5:4), I6=(2¯?0:1), I5=(3¯?1:0),I4=(3¯?3:2), I3=(4¯?1:0), I2=(4¯?1:0),
where the instruction ‘(v¯? l: h)’ means “If xv = 0, go to Il, otherwise go to Ih,” except that I_0 and I_1 unconditionally produce the values 0 and 1. We can convert any such branching program into a sequence of clauses, by translating ‘Ij=(v¯?l:h)’ into

(a¯j∨xv∨al)∧(a¯j∨x¯v∨ah),(173)

where a_0 is omitted, and where any clauses containing a_1 are dropped. We also omit a¯t, where It is the first instruction; in this example t = 8. (These simplifications correspond to asserting the unit clauses (a¯0)∧(a1)∧(at).) The branching program above therefore yields ten clauses,

(x1∨a7)∧(x¯1∨a6)∧(a¯7∨x2∨a5)∧(a¯7∨x¯2∨a4)∧(a¯6∨x2)∧(a¯5∨x¯3)∧(a¯4∨x3∨a3)∧(a¯4∨x¯3∨a2)∧(a¯3∨x¯4)∧(a¯2∨x4).

We can readily eliminate a_6, a_5, a_3, a_2, thereby getting a six-clause equivalent

(x1∨a7)∧(x¯1∨x2)∧(a¯7∨x2∨x¯3)∧(a¯7∨x¯2∨a4)∧(a¯4∨x3∨x¯4)∧(a¯4∨x¯3∨x4);
and a preprocessor will simplify this to the four-clause CNF

(x¯1∨x2)∧(x2∨x¯3)∧(x1∨x¯2∨x3∨x¯4)∧(x1∨x¯3∨x4),(174)

which appeared in exercise 7.1.1 19.

Exercise 417 explains why this translation scheme is valid. The method applies to any branching program whatsoever: The x variables can be tested in any order that is, the v’s need not be decreasing as in a BDD; moreover, a variable may be tested more than once.

Unit propagation and forcing

The effectiveness of an encoding depends largely on how well that encoding avoids bad partial assignments to the variables. If we’re trying to encode a Boolean condition f(x_1,x_2,...,xn), and if the tentative assignments x_1 ← 1 and x_2 ← 0 cause f to be false regardless of the values of x_3 through xn, we’d like the solver to deduce this fact without further ado, ideally by unit propagation once x_1 and x¯2 have been asserted. With a CDCL solver like Algorithm C, a quickly recognized conflict means a relatively short learned clause and that’s a hallmark of progress. Even better would be a situation in which unit propagation, after asserting x_1, would already force x_2 to be true; and furthermore if unit propagation after x¯2 would also force x¯1.

Such scenarios aren’t equivalent to each other. For example, consider the clauses F=(x¯1∨x3)∧(x¯1∨x2∨x¯3). Then, using the notation ‘F ⊦1 l’ to signify that F leads to l via unit propagation, we have F | x_1 ⊦1 x_2, but F | x¯2 ⊬1 x¯1. And with the clauses G=( x¯1∨x2∨x3)∧( x¯1∨x2∨x¯3) we have G|x1|x¯2⊦1ϵ (see Eq. (119)), but G | x_1 ⊬1 x_2 and G | x¯2 ⊦1 x¯1.

Consider now the simple at-most-one constraint on just three variables: f(x_1,x_2,x_3) = [x_1 + x_2 + x_3 ≤ 1]. We can try to represent f by proceeding methodically using the methods suggested above, either by constructing a circuit for f or by constructing f’s BDD. The first alternative (see exercise 420) yields

F=(x1∨x¯2∨a1)∧(x¯1∨x2∨a1)∧(x1∨x2∨a¯1)∧(x¯1∨x¯2)∧(x¯3∨a¯1);(175)

the second approach (see exercise 421) leads to a somewhat different solution,

G=(x1∨a4)∧(x¯1∨a3)∧(a¯4∨x¯2∨a2)∧(a¯3∨x2∨a2)∧(a¯3∨x¯2)∧(a¯2∨x¯3).(176)

But neither of these encodings is actually very good, because F | x3⊬1 x¯1 and G | x3⊬1 x¯1. Much better is the encoding that we get from the general scheme of (18) and (19) in the case n = 3, r = 1, namely

S=(a¯1∨a2)∧(x¯1∨a1)∧(x¯2∨a2)∧(x¯2∨a¯1)∧(x¯3∨a¯2),(177)

where a_1 and a_2 stand for s11 and s21; or the one obtained from (20) and (21),

B=(x¯3∨a1)∧(x¯2∨a1)∧(x¯2∨x¯3)∧(a¯1∨x¯1),(178)

where a_1 stands for b12. With either (177) or (178) we have S | xi ⊦1 x¯j and B | xi ⊦1 x¯j by unit propagation whenever i ≠ j. And of course the obvious encoding for this particular f is best of all, because n is so small:

O=(x¯1∨x¯2)∧(x¯1∨x¯3)∧(x¯2∨x¯3).(179)

Suppose f(x_1,...,xn) is a Boolean function that’s represented by a family of clauses F, possibly involving auxiliary variables {a_1,...,am}, as in (170). We say that F is a forcing representation if we have

F|L ⊦ limpliesF|L ⊦1 l(180)

whenever L ⋃ l is a set of strictly distinct literals contained in {x_1,...,xn, x¯1,...x¯n}. In other words, if the partial assignment represented by L logically implies the truth of some other literal l, we insist that unit propagation alone should be able to deduce l from F | L. The auxiliary variables {a_1,...,am} are exempt from this requirement; only the potential forcings between primary variables {x_1,...,xn} are supposed to be recognized easily when they occur.

(Technical point: If F | L ⊦ ϵ, meaning that F | L is unsatisfiable, we implicitly have F | L ⊦ l for all literals l. In such a case (180) tells us that F | L ⊦1 l and F⁢ |   L⊢1 l¯ both hold; hence F | L can then be proved unsatisfiable by unit propagation alone.)

We’ve seen that the clauses S and B in (177) and (178) are forcing for the constraint [x_1 + x_2 + x_3 ≤ 1], but the clauses F and G in (175) and (176) are not. In fact, the clauses of (18) and (19) that led to (177) are always forcing, for the general cardinality constraint [x_1 + … + xn ≤ r]; and so are the clauses of (20) and (21) that led to (178). (See exercises 429 and 430.) Moreover, the general at-most-one constraint [x_1 + … + xn ≤ 1] can be represented more efficiently by Heule’s 3(n−2) binary clauses and ⌊(n − 3)/2⌋ auxiliary variables (exercise 12), or with about n lg n binary clauses and only ⌈lg n⌉ auxiliary variables (exercise 394); both of those representations are forcing.

In general, we’re glad to know as soon as possible when a variable’s value has been forced by other values, because the variables of a large problem typically participate in many constraints simultaneously. If we know that x can’t be 0 in constraint f, then we can often conclude that some other variable y can’t be 1 in some other constraint g, if x appears in both f and g. There’s lots of feedback.

On the other hand it might be worse to use a large representation F that is forcing than to use a small representation G that isn’t, because additional clauses can make a SAT solver work harder. The tradeoffs are delicate, and they’re difficult to predict in advance.

Every Boolean constraint f(x_1,...,xn) has at least one forcing representation that involves no auxiliary variables. Indeed, it’s easy to see that the conjunctive prime form F of f the AND of all f’s prime clauses is forcing.

Smaller representations are also often forcing, even without auxiliaries. For example, the simple constraint [x_1 ≥ x_2 ≥ … ≥ xn] has (n2) prime clauses, namely (xj∨x¯k) for 1 ≤ j < k ≤ n; but only n − 1 of those clauses, the cases when k = j + 1 as in (164), are necessary and sufficient for forcing. Exercise 424 presents another, more-or-less random example.

In the worst case, all forcing representations of certain constraints are known to be huge, even when auxiliary variables are introduced (see exercise 428). But exercises 431 441 discuss many examples of useful and instructive forcing representations that require relatively few clauses.

We’ve glossed over an interesting technicality in definition (180), however: A sneaky person might actually construct a representation F that is absolutely useless in practice, even though it meets all of those criteria for forcing. For example, let G(a_1,...,am) be a family of clauses that are satisfiable but only when the auxiliary variables aj are set to extremely hard-to-find values. Then we might have f(x_1) = x_1 and F = (x_1) ∧ G(a_1,...,am)(!). This defect in definition (180) was first pointed out by M. Gwynne and O. Kullmann [arXiv:1406.7398 [cs.CC] (2014), 67 pages], who have also traced the history of the subject.

To avoid such a glitch, we implicitly assume that F is an honest representation of f, in the following sense: Whenever L is a set of n literals that fully characterizes a solution x_1 ...xn to the constraint f(x_1,...,xn) = 1, the clauses F | L must be easy to satisfy, using the SLUR algorithm of exercise 444. That algorithm is efficient because it does not backtrack. All of the examples in exercises 439 444 meet this test of honesty; indeed, the test is automatically passed whenever every clause of F contains at most one negated auxiliary variable.

Some authors have suggested that a SAT solver should branch only on primary variables xi, rather than on auxiliary variables aj, whenever possible. But an extensive study by M. Järvisalo and I. Niemelä [LNCS 4741 (2007), 348 363; J. Algorithms 63 (2008), 90 113] has shown that such a restriction is not advisable with Algorithm C, and it might lead to a severe slowdown.

Symmetry breaking

Sometimes we can achieve enormous speedup by exploiting symmetries. Consider, for example, the clauses for placing m + 1 pigeons into m holes, (106) (107). We’ve seen in Lemma B and Theorem B that Algorithm C and other resolution-related methods cannot demonstrate the unsatisfiability of those clauses without performing exponentially many steps as m grows. However, the clauses are symmetrical with respect to pigeons; independently, they’re also symmetrical with respect to holes: If π is any permutation of {0, 1,...,m} and if ρ is any permutation of {1, 2,...,m}, the transformation xjk ↦ x(jπ)(kρ) for 0 ≤ j ≤ m and 1 ≤ k ≤ m leaves the set of clauses (106) (107) unchanged. Thus the pigeonhole problem has (m + 1)! m! symmetries.

We’ll prove below that the symmetries on the holes allow us to assume safely that the hole-occupancy vectors are lexicographically ordered, namely that
x0kx1k …xmk ≤ x0(k+1)x1(k+1)⁢…xm(k+1),for 1≤k<m.(181)

These constraints preserve satisfiability; and we know from (169) that they are readily expressed as clauses. Without the help of such additional clauses the running time of Algorithm C rises from 19 megamems for m = 7 to 177 Mμ for m = 8, and then to 3.5 gigamems and 86 Gμ for m = 9 and 10. But with (181), the same algorithm shows unsatisfiability for m = 10 after only 1 megamem; and for m = 20 and m = 30 after only 284 Mμ and 3.6 Gμ, respectively.

Even better results occur when we order the pigeon-occupancy vectors:

xj1xj2 … xjm ≤ x(j+1)1x(j+1)2 ⁢… x(j+1)m,for 0≤j<m.(182)

With these constraints added to (106) and (107), Algorithm C polishes off the case m = 10 in just 69 kilomems. It can even handle m = 100 in 133 Mμ. This remarkable improvement was achieved by adding only m^2 − m new variables and 3m^2 − 2m new clauses to the original m^2 + m variables and (m + 1) + (m^3 + m^2)/2 clauses of (106) and (107). (Moreover, the reasoning that justifies (182) doesn’t “cheat” by invoking the mathematical pigeonhole principle behind the scenes.)

Actually that’s not all. The theory of columnwise symmetry (see exercise 498) also tells us that we’re allowed to add the (m2) simple binary clauses

(x(j−1)j∨x¯(j−1)k)for1≤j<k≤m(183)

to (106) and (107), instead of (182). This principle is rather weak in general; but it turns out to be ideally suited to pigeons: It reduces the running time for m = 100 to just 21 megamems, although it needs no auxiliary variables whatsoever!

Of course the status of (106) (107) has never been in doubt. Those clauses serve merely as training wheels because of their simplicity; they illustrate the fact that many symmetry-breaking strategies exist. Let’s turn now to a more interesting problem, which has essentially the same symmetries, but with the roles of pigeons and holes played by “points” and “lines” instead. Consider a set of m points and n lines, where each line is a subset of points; we will require that no two points appear together in more than one line. (Equivalently, no two lines may intersect in more than one point.) Such a configuration may be called quad-free, because it is equivalent to an m × n binary matrix (xij) that contains no “quad,” namely no 2 × 2 submatrix of 1s; element xij means that point i belongs to line j. Quad-free matrices are obviously characterized by (m2)(n2) clauses,

(x¯ij∨x¯ij′∨x¯i′j∨x¯i′j′),for 1≤i<i′≤m⁢  and 1≤j<j′≤n.(184)

What is the maximum number of 1s in an m × n quad-free matrix? [This question, when m = n, was posed by K. Zarankiewicz, Colloquium Mathematicæ 2 (1951), 301, who also considered how to avoid more general submatrices of 1s.] Let’s call that value Z(m, n) − 1; then Z(m, n) is the smallest r such that every m × n matrix with r nonzero entries contains a quad.

We’ve actually encountered examples of this problem before, but in a disguised form. For example (see exercise 448), a Steiner triple system on v objects exists if and only if v is odd and there is a quad-free matrix with m = v, n = v(v − 1)/6, and r = v(v − 1)/2. Other combinatorial block designs have similar characterizations.

Table 5 shows the values of Z(m, n) for small cases. These values were discovered by delicate combinatorial reasoning, without computer assistance; so it’s instructive to see how well a SAT solver can compete against real intelligence.

The first interesting case occurs when m = n = 8: One can place 24 markers on a chessboard without forming a quad, but Z(8, 8) = 25 markers is too many. If we simply add the cardinality constraints Σi=1m Σj=1n xij≥r to (184), Algorithm C will quickly find a solution when m = n = 8 and r = 24. But it bogs down when r = 25, requiring about 10 teramems to show unsatisfiability.

Fortunately we can take advantage of m! n! symmetries, which permute rows and columns without affecting quads. Exercise 495 shows that those symmetries allow us to add the lexicographic constraints

Table 5
Z(m, n), THE MINIMUM NUMBER OF 1S WITH (184) UNSATISFIABLE

n = 2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

m = 2:

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

m = 3:

5

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

m = 4:

6

8

10

11

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

m = 5:

7

9

11

13

15

16

18

19

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

m = 6:

8

10

13

15

17

19

20

22

23

25

26

28

29

31

32

33

34

35

36

37

38

39

40

41

42

43

m = 7:

9

11

14

16

19

22

23

25

26

28

29

31

32

34

35

37

38

40

41

43

44

45

46

47

48

49

m = 8:

10

12

15

18

20

23

25

27

29

31

33

34

36

37

39

40

42

43

45

46

48

49

51

52

54

55

m = 9:

11

13

16

19

22

25

27

30

32

34

37

38

40

41

43

44

46

47

49

50

52

53

55

56

58

59

m = 10:

12

14

17

21

23

26

29

32

35

37

40

41

43

45

47

48

50

52

53

55

56

58

59

61

62

64

m = 11:

13

15

18

22

25

28

31

34

37

40

43

45

46

48

51

52

54

56

58

60

61

63

64

66

67

69

m = 12:

14

16

19

23

26

29

33

37

40

43

46

49

50

52

54

56

58

61

62

64

66

67

69

71

73

74

m = 13:

15

17

20

24

28

31

34

38

41

45

49

53

54

56

58

60

62

65

67

68

80

72

74

76

79

80

m = 14:

16

18

21

25

29

32

36

40

43

46

50

54

57

59

61

64

66

69

71

73

74

76

79

81

83

85

m = 15:

17

19

22

26

31

34

37

41

45

48

52

56

59

62

65

68

70

73

76

78

79

81

83

86

87

89

m = 16:

18

20

23

27

32

35

39

43

47

51

54

58

61

65

68

71

74

77

81

82

84

86

88

91

92

94

[References: R. K. Guy, in Theory of Graphs, Tihany 1966, edited by Erdős and Katona (Academic Press, 1968), 119 150; R. J. Nowakowski, Ph.D. thesis (Univ. of Calgary, 1978), 202.]

xi1xi2⋅⋅⋅xin≥x(i+1)1x(i+1)2⋅⋅⋅x(i+1)n,for 1≤i<m;(185)

x1jx2j⋅⋅⋅xmj≥x1(j+1)x2(j+1)⋅⋅⋅xm(j+1),for 1≤j<n.(186)

(Increasing order, with ‘≤’ in place of ‘≥’, could also have been used, but decreasing order turns out to be better; see exercise 497.) The running time to prove unsatisfiability when r = 25 now decreases dramatically, to only about 50 megamems. And it falls to 48 Mμ if the lexicographic constraints are shortened to consider only the leading 4 elements of a row or column, instead of testing all 8.

The constraints of (185) and (186) are useful in satisfiable problems too not in the easy case m = n = 8, when they aren’t necessary, but for example in the case m = n = 13 when r = 52: Then they lead Algorithm C to a solution after about 200 gigamems, while it needs more than 18 teramems to find a solution without such help. (See exercise 449.)

Satisfiability-preserving maps

Let’s proceed now to the promised theory of symmetry breaking. In fact, we will do more: Symmetry is about permutations that preserve structural properties, but we will consider arbitrary mappings instead. Mappings are more general than permutations, because they needn’t be invertible. If x = x_1 ... xn is any potential solution to a satisfiability problem, our theory is based on transformations τ that map x↦xτ=x1′…xn′, where xτ is required to be a solution whenever x is a solution.

In other words, if F is a family of clauses on n variables and if f(x) = [x satisfies F], then we are interested in all mappings τ for which f(x) ≤ f(xτ). Such a mapping is conventionally called an endomorphism of the solutions.[*] If an endomorphism τ is actually a permutation, it’s called an automorphism. Thus, if there are K solutions to the problem, out of N = 2n possibilities, the total number of mappings is NN; the total number of endomorphisms is KK NN−K; and the total number of automorphisms is K!(N − K)!.

* This word is a bit of a mouthful. But it’s easier to say “endomorphism” than to say “satisfiability-preserving transformation,” and you can use it to impress your friends. The term “conditional symmetry” has also been used by several authors in special cases.

Notice that we don’t require f(x) to be exactly equal to f(xτ). An endomorphism is allowed to map a nonsolution into a solution, and only KK (N − K)N−K mappings satisfy that stronger property. On the other hand, automorphisms always do satisfy f(x)= f(xτ); see exercise 454.

Here, for instance, is a more-or-less random mapping when n = 4:

Exercises 455 and 456 discuss potential endomorphisms of this mapping.

In general there will be one or more cycles, and every element of a cycle is the root of an oriented tree that leads to it. For example, the cycles of (187) are (0011), (1010 0101 0110), and (1000).

Several different endomorphisms τ_1, τ_2, ... , τp are often known. In such cases it’s helpful to imagine the digraph with 2n vertices that has arcs from each vertex x to its successors xτ_1, xτ_2, ... , xτp. This digraph will have one or more sink components, which are strongly connected components Y from which there is no escape: If x ∈ Y then xτk ∈ Y for 1 ≤ k ≤ p. (In the special case where each τk is an automorphism, the sink components are traditionally called orbits of the automorphism group.) When p = 1, a sink component is the same as a cycle.

The clauses F are satisfiable if and only if f(x) = 1 for at least one x. Such an x will lead to at least one sink component Y, all of whose elements will satisfy f(y) = 1. Thus it suffices to test satisfiability by checking just one element y in every sink component Y, to see if f(y) = 1.

Let’s consider a simple problem based on the “sweep” of an m×n matrix X = (xij), which is the largest diagonal sum of any t × t submatrix:

sweep(X)=max1≤i1<i2<…<it≤m1≤j1<j2<…<jt≤n(xi1j1+xi2j2+…+xitjt).(188)

When X is binary, sweep(X) is the length of the longest downward-and-rightward path that passes through its 1s. We can use satisfiability to decide whether such a matrix exists having sweep(X) ≤ k and Σi=1m Σj=1n xij≥r, given m, n, k, and r; suitable clauses are exhibited in exercise 460. A solution with m = n = 10, k = 3, and r = 51 appears at the right: It has 51 1s, but no four of them lie in a monotonic southeasterly path.

0000111111
0000100011
0000100111
0001101101
0111111001
1111100001
1010000011
1010000010
1110111110
1111100000

This problem has 2mn candidate matrices X, and experiments with small m and n suggest several endomorphisms that can be applied to such candidates without increasing the sweep.

- τ_1: If xij = 1 and xi(j + 1) = 0, and if xi′j = 0 for 1 ≤ i′ < i, we can set xij ← 0 and xi(j+1) ← 1.

- τ_2: If xij = 1 and x(i+1)j = 0, and if xij′ = 0 for 1 ≤ j′ < j, we can set xij ← 0 and x(i+1)j ← 1.

- τ_3: If the 2 × 2 submatrix in rows {i, i + 1} and columns {j, j + 1} is 1110, we can change it to 0111.

These transformations are justified in exercise 462. They’re sometimes applicable for several different i and j; for instance, τ_3 could be used to change any of eight different 2×2 submatrices in the example solution. In such cases we make an arbitrary decision, by choosing (say) the lexicographically smallest possible i and j.

The clauses that encode this problem have auxiliary variables besides xij; but we can ignore the auxiliary variables when reasoning about endomorphisms.

Each of these endomorphisms either leaves X unchanged or replaces it by a lexicographically smaller matrix. Therefore the sink components of {τ_1, τ_2, τ_3} consist of the matrices X that are fixed points of all three transformations. Hence we’re allowed to append additional clauses, stating that neither τ_1 nor τ_2 nor τ_3 is applicable. For instance, transformation τ_3 is ruled out by the clauses

∧i=1m−1∧j=1n−1(x¯ij∨x¯j(j+1)∨x(i+1)(j+1),(189)

which state that the submatrix 1110 doesn’t appear. The clauses for τ_1 and τ_2 are only a bit more complicated (see exercise 461).

These additional clauses give interesting answers in satisfiable instances, although they aren’t really helpful running-time-wise. On the other hand, they’re spectacularly successful when the problem is unsatisfiable.

For example, we can show, without endomorphisms, that the case m = n = 10, k = 3, r = 52 is impossible, and hence that any solution for r = 51 is optimum; Algorithm C proves this after about 16 gigamems of work. Adding the clauses for τ_1 and τ_2, but not τ_3, increases the running time to 23 Gμ; on the other hand the clauses for τ_3 without τ_1 or τ_2 reduce it to 6 Gμ. When we use all three endomorphisms simultaneously, however, the running time to prove unsatisfiability goes down to just 3.5 megamems, a speedup of more than 4500.

Even better is the fact that the fixed points of {τ_1, τ_2, τ_3} actually have an extremely simple form — see exercise 463 — from which we can readily determine the answer by hand, without running the machine at all! Computer experiments have helped us to guess this result; but once we’ve proved it, we’ve solved infinitely many cases in one fell swoop. Theory and practice are synergistic.

Another interesting example arises when we want to test whether or not a given graph has a perfect matching, which is a set of nonoverlapping edges that exactly touch each vertex. We’ll discuss beautiful, efficient algorithms for this problem in Sections 7.5.1 and 7.5.5; but it’s interesting to see how well a simple-minded SAT solver can compete with those methods.

Perfect matching is readily expressible as a SAT problem whose variables are called ‘uv’, one for each edge u — v. Variables ‘uv’ and ‘vu’ are identical. Whenever the graph contains a 4-cycle v_0 — v_1 — v_2 — v_3 — v_0, we might include two of its edges {v_0v_1, v_2v_3} in the matching; but we could equally well have included {v_1v_2, v_3v_0} instead. Thus there’s an endomorphism that says, “If v_0v_1 = v_2v_3 = 1 (hence v_1v_2 = v_3v_0 = 0), set v_0v_1 ← v_2v_3 ← 0 and v_1v_2 ← v_3v_0 ← 1.”

And we can carry this idea further: Let the edges be totally ordered in some arbitrary fashion, and for each edge uv consider all 4-cycles in which uv is the largest edge. In other words, we consider all cycles of the form u — v — u′ — v′ — u in which vu′, u′ v′, v′ u all precede uv in the ordering. If any such cycles exist, choose one of them arbitrarily, and let τuv be one of two endomorphisms:

τuv−:"Ifuv=u′v′=1,set  uv←u′v′←0and vu′←v′u←1."τuv+:"Ifvu′=v′u=1,set  uv←u′v′←1and vu′←v′u←0."

Either τuv− or τuv+ is stipulated, for each uv. Exercise 465 proves that a perfect matching is in the sink component of any such family of endomorphisms if and only if it is fixed by all of them. Therefore we need only search for fixed points.

For example, consider the problem of covering an m×n board with dominoes. This is the problem of finding a perfect matching on the grid graph Pm ⎕ Pn. The graph has mn vertices (i, j), with m(n − 1) “horizontal” edges hij from (i, j) to (i, j + 1) and (m − 1)n “vertical” edges vij from (i, j) to (i + 1, j). It has exactly (m − 1)(n − 1) 4-cycles; and if we number the edges from left to right, no two 4-cycles have the same largest edge. Therefore we can construct (m − 1)(n − 1) endomorphisms, in each of which we’re free to decide whether to allow a particular cycle to be filled by two horizontal dominoes or by two vertical ones.

Let’s stipulate that hij and h(i+1)j are allowed together only when i + j is odd; vij and vi(j+1) are allowed together only when i + j is even. The nine endomorphisms when m = n = 4 are then

And it’s not difficult to see that only one 4 × 4 domino covering is fixed by all nine. Indeed (exercise 466), the solution turns out to be unique for all m and n.

The famous problem of the “mutilated chessboard” asks for a domino covering when two opposite corner cells have been removed. This problem is unsatisfiable when m and n are both even, by exercise 7.1.4 213. But a SAT solver can’t discover this fact quickly from the clauses alone, because there are many ways to get quite close to a solution; see the discussion following 7.1.4 (130). [S. Dantchev and S. Riis, in FOCS 42 (2001), 220 229, have proved in fact that every resolution refutation of these clauses requires 2Ω(n) steps.]

When Algorithm C is presented with mutilated boards of sizes 6 × 6, 8 × 8, 10 × 10, ... , 16 × 16, it needs respectively about 55 Kμ, 1.4Mμ, 31Mμ, 668 Mμ, 16.5Gμ, and .91 Tμ (that’s teramems) to prove unsatisfiability. The even-odd endomorphisms typified by (190) come to our rescue, however: They narrow the search space spectacularly, reducing the respective running times to only 15 Kμ, 60 Kμ, 135 Kμ, 250 Kμ, 470 Kμ, 690 Kμ (that’s kilomems). They even can verify the unsatisfiability of a mutilated 256×256 domino cover after fewer than 4.2Gμ of calculation, exhibiting a growth rate of roughly O(n^3).

Endomorphisms can also speed up SAT solving in another important way:

Theorem E. Let p_1p_2...pn be any permutation of {1, 2,...,n}. If the Boolean function f(x_1,x_2,...,xn) is satisfiable, then it has a solution such that xp_1xp_2...xpn is lexicographically less than or equal to xp1′xp2′…xpn′ for every endomorphism of f that takes x1x2…xn↦x1′x2′…xn′.

Proof. The lexicographically smallest solution of f has this property.

Maybe we shouldn’t call this a “theorem”; it’s an obvious consequence of the fact that endomorphisms always map solutions into solutions. But it deserves to be remembered and placed on some sort of pedestal, because we will see that it has many useful applications.

Theorem E is extremely good news, at least potentially, because every Boolean function has a huge number of endomorphisms. (See exercise 457.) On the other hand, there’s a catch: We almost never know any of those endomorphisms until after we’ve solved the problem! Still, whenever we do happen to know one of the zillions of nontrivial endomorphisms that exist, we’re allowed to add clauses that narrow the search. There’s always a “lex-leader” solution that satisfies x1x2…xn≤x1′x2′…xn′, if there’s any solution at all.

A second difficulty that takes some of the shine away from Theorem E is the fact that most endomorphisms are too complicated to express neatly as clauses. What we really want is an endomorphism that’s nice and simple, so that lexicographic ordering is equally simple.

Fortunately, such endomorphisms are often available; in fact, they’re usually automorphisms—symmetries of the problem—defined by signed permutations of the variables. A signed permutation represents the operation of permuting variables and/or complementing them; for example, the signed permutation ‘4¯132¯’ stands for the mapping (x1,x2,x3,x4)↦(x4¯,x1,x3,x2¯)=(x¯4,x1,x3,x¯2). This operation transforms the states in a much more regular way than (187):

If σ takes the literal u into v, we write uσ = v; and in such cases σ also takes u¯ into v¯. Thus we always have u¯σ=uσ¯. We also write xσ for the result of applying σ to a sequence x of literals; for example, (x1,x2,x3,x4)σ=(x¯4,x1,x3,x¯2). This mapping is a symmetry or automorphism of f(x) if and only if f(x) = f(xσ) for all x. Exercises 474 and 475 discuss basic properties of such symmetries; see also exercise 7.2.1.2 20.

Notice that a signed permutation can be regarded as an unsigned permutation of the 2n literals {x1,…,xn,x¯1,…,x¯n}, and as such it can be written as a product of cycles. For instance, the symmetry 4¯132¯ corresponds to the cycles (14¯2)(1¯42¯)(3)(3¯). We can multiply signed permutations by multiplying these cycles in the normal way, just as in Section 1.3.3.

The product στ of two symmetries σ and τ is always a symmetry. Thus in particular, if σ is any symmetry, so are its powers σ^2, σ^3, etc. We say that σ has order r if σ, σ^2, ... , σr are distinct and σr is the identity. A signed permutation of order 1 or 2 is called a signed involution; this important special case arises if and only if σ is its own inverse (σ^2 = 1).

It’s clearly easier to work with permutations of 2n literals than to work with permutations of 2n states x_1 ... xn. The main advantage of a signed permutation σ is that we can test whether or not σ preserves the family F of clauses in a satisfiability problem. If it does, we can be sure that σ also is an automorphism when it acts on all 2n states. (See exercise 492.)

Let’s go back to the example waerden (3, 10; 97) that we’ve often discussed above. These clauses have an obvious symmetry, which takes x_1x_2 ... x_97 ↦ x_97x_96 ... x_1. If we don’t break this symmetry, Algorithm C typically verifies unsatisfiability after about 530 Mμ of computation. Now Theorem E tells us that we can also assert that x_1x_2x_3 ≤ x_97x_96x_95, say; but that symmetry-breaker doesn’t really help at all, because x_1 has very little influence on x_97. Fortunately, however, Theorem E allows us to choose any permutation p_1p_2 ... pn on which to base lexicographic comparisons. For example, we can assert that x_48x_47x_46 ... ≤ x_50x_51x_52 ... provided that we don’t also require x_1x_2x_3 ... ≤ x_97x_96x_95 ... . (One fixed global ordering must be used, but the endomorphs can be arbitrary.)

Even the simple assertion that x_48 ≤ x_50, which is the clause ‘48¯⁢ 50’, cuts the running time down to about 410 Mμ, because this new clause combines nicely with the existing clauses 46 48 50, 48 49 50, 48 50 52 to yield the helpful binary clauses 46 50, 49 50, 50 52. If we go further and assert that x_48x_47 ≤ x_50x_51, the running time improves to 345 Mμ. And the next steps x_48x_47x_46 ≤ x_50x_51x_52, ... , x_48x_47x_46x_45x_44x_43 ≤ x_50x_51x_52x_53x_54x_55 take us down to 290 Mμ, then 260 Mμ, 235 Mμ, 220 Mμ; we’ve saved more than half of the running time by exploiting a single reflection symmetry! Only 16 simple additional clauses, namely

48¯⁢ 50, 48¯ a1, 50 a1, 47¯ 51 a¯1, 47¯ a2 a¯1, 51 a2 a¯1, 46¯ 52 a¯2, …,43¯ 55 a¯5
are needed to get this speedup, using the efficient encoding of lex order in (169).

Of course all good things come to an end, and we’ve now reached the point of diminishing returns: Further clauses to assert that x_48x_47 ... x_42 ≤ x_50x_51 ... x_56 in the waerden (3, 10; 97) problem turn out to be counterproductive.

A wonderful simplification occurs when a symmetry σ is a signed involution that has comparatively few 2-cycles. Suppose, for example, that σ=53¯2¯416¯9¯8¯7¯; in cycle form this is (15)(1¯5¯)(23¯)(2¯3)(4)(4¯)(66¯)(79¯)(7¯9)(88¯). Then the lexicographic relation x=x1…x9≤x1′…x9′=xσ holds if and only if x1x2x6≤x5x¯3x¯6. The reason is clear, once we look closer (see F. A. Aloul, A. Ramani, I. L. Markov, and K. A. Sakallah, IEEE Trans. CAD-22 (2003), 1117 1137, §III.C): The relation x1…x9≤x1′…x9′ means, in this case, “x_1 ≤ x_5; if x_1 = x_5 then x2≤x¯3; if x_1 = x_5 and x2=x¯3 then x3≤x¯2; if x_1 = x_5, x2=x¯3, and x3=x¯2 then x_4 ≤ x_4; if x_1 = x_5, x2=x¯3, x3=x¯2, and x_4 = x_4 then x_5 ≤ x_1; if x_1 = x_5, x2=x¯3, x3=x¯2, x_4 = x_4, and x_5 = x_1 then x6≤x¯6; if x_1 = x_5, x2=x¯3, x3=x¯2, x_4 = x_4, x_5 = x_1, and x6=x¯6 then we’re done for.” With this expanded description the simplifications are obvious.

In general this reasoning allows us to improve Theorem E as follows:

Corollary E. Let p_1p_2 ... pn be any permutation of {1, 2,..., n}. For every signed involution σ that is a symmetry of clauses F, we can write σ in cycle form

(pi1±pj1)(p¯i1∓pj1)(pi2±pj2)(p¯i2∓pj2)…(pit±pjt)(p¯it∓pjt)(192)

with i_1 ≤ j_1, i_2 ≤ j_2, ... , it ≤ jt, i_1 < i_2 < … < it, and with (p¯ik∓pjk) omitted when ik = jk; and we’re allowed to append clauses to F that assert the lexicographic relation xpi1xpi2...xpiq≤x±pj1x±pj2...x±pjq, where q = t or q is the smallest k with ik = jk.

In the common case when σ is an ordinary signless involution, all of the signs can be eliminated here; we simply assert that xpi1...xpit≤xpj1...xpjt.

This involution principle justifies all of the symmetry-breaking techniques that we used above in the pigeonhole and quad-free matrix problems. See, for example, the details discussed in exercise 495.

The idea of breaking symmetry by appending clauses was pioneered by J.-F. Puget [LNCS 689 (1993), 350–361], then by J. Crawford, M. Ginsberg, E. Luks, and A. Roy [Int. Conf. Knowledge Representation and Reasoning 5 (1998), 148–159], who considered unsigned permutations only. They also attempted to discover symmetries algorithmically from the clauses that were given as input. Experience has shown, however, that useful symmetries can almost always be better supplied by a person who understands the structure of the underlying problem.

Indeed, symmetries are often “semantic” rather than “syntactic.” That is, they are symmetries of the underlying Boolean function, but not of the clauses themselves. In the Zarankiewicz problem about quad-free matrices, for example, we appended efficient cardinality clauses to ensure that ∑xij ≥ r; that condition is symmetric under row and column swaps, but the clauses are not.

In this connection it may also be helpful to mention the monkey wrench principle: All of the techniques by which we’ve proved quickly that the pigeonhole clauses are unsatisfiable would have been useless if there had been one more clause such as (x01∨x11∨x¯22); that clause would have destroyed the symmetry!

We conclude that we’re allowed to remove clauses from F until reaching a subset of clauses F_0 for which symmetry-breakers S can be added. If F = F_0∪F_1, and if F_0 is satisfiable ⇔ F_0 ∪ S is satisfiable, then F_0 ∪ S ⊦ ∈ ⇒ F ⊦ ∈.

One hundred test cases

And now—ta da!—let’s get to the climax of this long story, by looking at how our SAT solvers perform when presented with 100 moderately challenging instances of the satisfiability problem. The 100 sets of clauses summarized on the next two pages come from a cornucopia of different applications, many of which were discussed near the beginning of this section, while others appear in the exercises below.

Every test case has a code name, consisting of a letter and a digit. Table 6 characterizes each problem and also shows exactly how many variables, clauses, and total literals are involved. For example, the description of problem A1 ends with ‘2043|24772¯|55195|U’; this means that A1 consists of 24772 clauses on 2043 variables, having 55195 literals altogether, and those clauses are unsatisfiable. Furthermore, since ‘24772’ is underlined, all of A1’s clauses have length 3 or less.

Table 6
CAPSULE SUMMARIES OF THE HUNDRED TEST CASES

A1. Find x = x_1x_2 ... x_99 with νx = 27 and no three equally spaced 1s. (See exercise 31.)

2043|24772|55195|U

A2. Like A1, but x_1x_2 ... x_100.

2071|25197|56147|S

B1. Cover a mutilated 10 × 10 board with 49 dominoes, without using extra clauses to break symmetry.

176|572|1300|U

B2. Like B1, but a 12 × 12 board with 71 dominoes.

260|856|1948|U

C1. Find an 8-step Boolean chain that computes (z_2z_1z_0)2 = x_1 + x_2 + x_3 + x_4. (See exercise 479(a).)

384|16944|66336|U

C2. Find a 7-step Boolean chain that computes the modified full adder functions z_1, z_2, z_3 in exercise 481(b).

469|26637|100063|U

C3. Like C2, but with 8 steps.

572|33675|134868|S

C4. Find a 9-step Boolean chain that computes zl and zr in the mod-3 addition problem of exercise 480(b).

678|45098|183834|S

C5. Connect A to A, ... , J to J in Dudeney’s puzzle of exercise 392, (iv).

1980|22518|70356|S

C6. Like C5, but move the J in row 8 from column 4 to column 5.

1980|22518|70356|U

C7. Given binary strings s_1, ... , s_50 of length 200, randomly generated at distances ≤ rj from some string x, find x (see exercise 502).

65719|577368|1659623|S

C8. Given binary strings s_1, ... , s_40 of length 500, inspired by biological data, find a string at distance ≤ 42 from each of them.

123540|909120|2569360|U

C9. Like C8, but at distance ≤ 43.

124100|926200|2620160|S

D1. Satisfy factor fifo (18, 19, 111111111111). (See exercise 41.)

1940|6374|16498|U

D2. Like D1, but factor lifo .

1940|6374|16498|U

D3. Like D1, but (19, 19, 111111111111).

2052|6745|17461|S

D4. Like D2, but (19, 19, 111111111111).

2052|6745|17461|S

D5. Solve (x_1 ... x_9)2 × (y_1 ... y_9)2 ≠ (x_1 ... x_9)2 × (y_1 ... y_9)2, with two copies of the same Dadda multiplication circuit.

864|2791|7236|U

E0. Find an Erd˝os discrepancy pattern x_1 ... x_500 (see exercise 482).

1603|9157|27469|S

E1. Like E0, but x_1 ... x_750.

2556|14949|44845|S

E2. Like E0, but x_1 ... x_1000.

3546|21035|63103|S

F1. Satisfy fsnark (99). (See exercise 176.)

1782|4161|8913|U

F2. Like F1, but without the clauses (e¯1,3∨f¯99,3)∧(f¯1,1∨e¯2,1).

1782|4159|8909|S

G1. Win Late Binding Solitaire with the “most difficult winnable deal” in answer 486.

1242|22617|65593|S

G2. Like G1, but with the most difficult unwinnable deal.

1242|22612|65588|U

G3. Find a test pattern for the fault “B4343 stuck at 0” in prod (16, 32).

3498|11337|29097|S

G4. Like G3, but for the fault “D3413,9 stuck at 0.”

3502|11349|29127|S

G5. Find a 7 × 15 array X_0 leading to X_3 = 𝗟𝗜𝗞𝗘 as in Fig. 78, having at most 38 live cells.

7150|28508|71873|U

G6. Like G5, but at most 39 live cells.

7152|28536|71956|S

G7. Like G5, but X_4 = 𝗟𝗜𝗞𝗘 and X_0 can be arbitrary.

8725|33769|84041|U

G8. Find a configuration in the Game of Life that proves f∗(7, 7) = 28 (see exercise 83).

97909|401836|1020174|S

K0. Color the 8 × 8 queen graph with 8 colors, using the direct encoding (15) and (16), also forcing the colors of all vertices in the top row.

512|5896|12168|U

K1. Like K0, but with the exclusion clauses (17) also.

512|7688|15752|U

K2. Like K1, but with kernel clauses instead of (17) (see answer 14).

512|6408|24328|U

K3. Like K1, but with support clauses instead of (16) (see exercise 399).

512|13512|97288|U

K4. Like K1, but using the order encoding for colors.

448|6215|21159|U

K5. Like K4, but with the hint clauses (162) appended.

448|6299|21663|U

K6. Like K5, but with double clique hints (exercise 396).

896|8559|27927|U

K7. Like K1, but with the log encoding of exercise 391(a).

2376|5120|15312|U

K8. Like K1, but with the log encoding of exercise 391(b).

192|5848|34968|U

L1. Satisfy langford (10).

130|2437|5204|U

L2. Satisfy langford′(10).

273|1020|2370|U

L3. Satisfy langford (13).

228|5875|12356|U

L4. Satisfy langford′(13).

502|1857|4320|U

L5. Satisfy langford (32).

1472|102922|210068|S

L6. Satisfy langford′(32).

3512|12768|29760|S

L7. Satisfy langford (64).

6016|869650|1756964|S

L8. Satisfy langford′(64).

14704|53184|124032|S

M1. Color the McGregor graph of order 10 (Fig. 76) with 4 colors, using one color at most 6 times, via the cardinality constraints (18) and (19).

1064|2752|6244|U

M2. Like M1, but via (20) and (21).

814|2502|5744|U

M3. Like M1, but at most 7 times.

1161|2944|6726|S

M4. Like M2, but at most 7 times.

864|2647|6226|S

M5. Like M4, but order 16 and at most 11 times.

2256|7801|18756|U

M6. Like M5, but at most 12 times.

2288|8080|19564|S

M7. Color the McGregor graph of order 9 with 4 colors, and with at least 18 regions doubly colored (see exercise 19).

952|4539|13875|S

M8. Like M7, but with at least 19 regions.

952|4540|13877|U

N1. Place 100 nonattacking queens on a 100 × 100 board.

10000|1151800|2313400|S

O1. Solve a random open shop scheduling problem with 8 machines and 8 jobs, in 1058 units of time.

50846|557823|1621693|U

O2. Like O1, but in 1059 units.

50901|558534|1623771|S

P0. Satisfy (99), (100), and (101) for m = 20, thereby exhibiting a poset of size 20 with no maximal element.

400|7260|22080|U

P1. Like P0, but with m = 14 and using only the clauses of exercise 228.

196|847|2667|U

P2. Like P0, but with m = 12 and using only the clauses of exercise 229.

144|530|1674|U

P3. Like P2, but omitting the clause (x¯31∨x¯16∨x36).

144|529|1671|S

P4. Like P3, but with m = 20.

400|2509|7827|S

Q0. Like K0, but with 9 colors.

576|6624|13688|S

Q1. Like K1, but with 9 colors.

576|8928|18296|S

Q2. Like K2, but with 9 colors.

576|7200|27368|S

Q3. Like K3, but with 9 colors.

576|15480|123128|S

Q4. Like K4, but with 9 colors.

512|7008|24200|S

Q5. Like K5, but with 9 colors.

512|7092|24704|S

Q6. Like K6, but with 9 colors.

1024|9672|31864|S

Q7. Like K7, but with 9 colors.

3168|6776|20800|S

Q8. Like K8, but with 9 colors.

256|6776|52832|S

Q9. Like Q8, but with the log encoding of exercise 391(c).

256|6584|42256|S

R1. Satisfy rand (3, 1061, 250, 314159).

250|1061|3183|S

R2. Satisfy rand (3, 1062, 250, 314159).

250|1062|3186|U

S1. Find a 4-term disjunctive normal form on {x_1,...,x_20} that differs from (27) but agrees with it at 108 random training points.

356|4229|16596|S

S2. Like S1, but at 109 points.

360|4310|16760|U

S3. Find a sorting network on nine elements that begins with the comparators [1:6][2:7][3:8][4:9] and finishes in five more parallel rounds. (See exercise 64.)

5175|85768|255421|U

S4. Like S3, but in six more rounds.

6444|107800|326164|S

T1. Find a 24 × 100 tatami tiling that spells ‘TATAMI’ as in exercise 118.

2874|10527|26112|S

T2. Like T1, but 24 × 106 and the ‘I’ should have serifs.

3048|11177|27724|U

T3. Solve the TAOCP problem of exercise 389 with only 4 knight moves.

3752|12069|27548|U

T4. Like T3, but with 5 knight moves.

3756|12086|27598|S

T5. Find the pixel in row 5, column 18 of Fig. 80(c), the lexicographically last solution to the Cheshire Tom problem.

8837|39954|100314|S

T6. Like T5, but column 19.

8837|39955|100315|U

T7. Solve the run-count extension of the Cheshire Tom problem (see exercise 117).

25734|65670|167263|S

T8. Like T7, but find a solution that differs from Fig. 79.

25734|65671|167749|U

W1. Satisfy waerden (3, 10; 97).

97|2779|11662|U

W2. Satisfy waerden (3, 13; 159).

159|7216|31398|S

W3. Satisfy waerden (5, 5; 177).

177|7656|38280|S

W4. Satisfy waerden (5, 5; 178).

178|7744|38720|U

X1. Prove that the “taking turns” protocol (43) gives mutual exclusion for at least 100 steps.

1010|3612|10614|U

X2. Prove that assertions Φ for the four-bit protocol of exercise 101, analogous to (50), are invariant.

129|354|926|U

X3. Prove that Bob won’t starve in 36 steps, assuming the Φ of X2.

1652|10552|28971|U

X4. Prove that there’s a simple 36-step path with the four-bit protocol, assuming the Φ of X2.

22199|50264|130404|S

X5. Like X4, but 37 steps.

23388|52822|137034|U

X6. Like X1, but with Peterson’s protocol (49) instead of (43).

2218|8020|23222|U

X7. Prove that there’s a simple 54-step path with protocol (49).

26450|56312|147572|S

X8. Like X7, but 55 steps.

27407|58317|152807|U

Fig. 95. The clauses of these test cases bind the variables together in significantly different ways. (Illustrations by Carsten Sinz.)

Of course we can’t distinguish hard problems from easy ones by simply counting variables, clauses, and literals. The great versatility with which clauses can capture logical relationships means that different sets of clauses can lead to wildly different phenomena. Some of this immense variety is indicated in Fig. 95, which depicts ten instructive “variable interaction graphs.” Each variable is represented by a ball, and two variables are linked when they appear together in at least one clause. (Some edges are darker than others; see exercise 506. For further examples of such 3D visualizations, presented also in color, see Carsten Sinz, Journal of Automated Reasoning 39 (2007), 219–243.)

A single SAT solver cannot be expected to excel on all of the many species of problems. Furthermore, nearly all of the 100 instances in Table 6 are well beyond the capabilities of the simple algorithms that we began with: Algorithms A, B, and D are unable to crack any of those test cases without needing more than fifty gigamems of computation, except for the simplest examples—L1, L2, L5, P3, P4, and X2. Algorithm L, the souped-up refinement of Algorithm D, also has a lot of difficulty with most of them. On the other hand, Algorithm C does remarkably well. It polishes off 79 of the given problems in fewer than ten Gμ.

Thus the test cases of Table 6 are tough, yet they’re within reach. Almost all of them can be solved in say two minutes, at most, with methods known today.

Complete details can be found in the file SATexamples.tgz on the author’s website, together with many related problems both large and small.

Exactly 50 of these 100 cases are satisfiable. So we’re naturally led to wonder whether Algorithm W (“WalkSAT”) will handle such cases well. The answer is that Algorithm W sometimes succeeds brilliantly—especially on problems C7, C9, L5, L7, M3, M4, M6, P3, P4, Q0, Q1, R1, S1, where it typically outperforms all the other methods we’ve discussed. In particular it solved S1 in just 1 Mμ, in the author’s tests with N = 50n and p = .4, compared to 25 Mμ by the next best method, Algorithm C; it won by 15 Mμ versus Algorithm C’s 83 Mμ on M3, by 83 Mμ versus Algorithm L’s 104 Mμ on Q0, by 95 Mμ versus Algorithm C’s 464 Mμ on Q1, and by a whopping 104 Mμ versus Algorithm C’s 7036 Mμ on C7. That was a surprise. WalkSAT also was reasonably competitive on problem N1. But in all other cases it was nowhere near the method of choice. Therefore we’ll consider only Algorithms L and C in the remainder of this discussion.[*]

When does a lookahead algorithm like Algorithm L outperform a clause-learning algorithm like Algorithm C? Figure 96 shows how they compare to each other on our 100 test cases: Each problem is plotted with Algorithm C’s running time on the vertical axis and Algorithm L’s on the horizontal axis. Thus Algorithm L is the winner for problems that appear above the dotted line. (This dotted line is “wavy” because times aren’t drawn to scale: The kth fastest running time is shown as k units from the left of the page or from the bottom.)

* There actually are two variants of Algorithm L, because the alternative heuristics of exercise 143 must be used for looking ahead when clauses of length 4 or more are present. We could use exercise 143 even when given all-ternary clauses; but experience shows that we’d tend to lose a factor of 2 or more by doing so. Our references to Algorithm L therefore implicitly assume that exercise 143 is being applied only when necessary.

Fig. 96. Comparison of Algorithms C and L on 100 moderately difficult satisfiability problems.

All of these experiments were aborted after 50 Gμ, if necessary, since many of these problems could potentially take centuries before running to completion. Thus the test cases for which Algorithm L timed out appear at the right edge of Fig. 96, and the tough cases for Algorithm C appear at the top. Only E2 and X8 were too hard for both algorithms to handle within the specified cutoff time.

Algorithm L is deterministic: It uses no random variables. However, a slight change (see exercise 505) will randomize it, because the inputs can be shuffled as they are in Algorithm C; and we might as well assume that this change has been made. Then both Algorithms L and C have variable running times. They will find solutions or prove unsatisfiability more quickly on some runs than on others, as we’ve already seen for Algorithm C in Fig. 92.

To compensate for this variability, each of the runtimes reported in Fig. 96 is the median of nine independent trials. Figure 97 shows all 9×100 of the empirical running times obtained with Algorithm C, sorted by their median values. We can see that many of the problems have near-constant behavior; indeed, the ratio max/min was less than 2 in 38 of the cases. But 10 cases turned out to be highly erratic in these experiments, with max/min > 100; problem P4 was actually solved once after only 323 kilomems, while another run lasted 339 gigamems!

One might expect satisfiable problems, such as P4, to benefit more from lucky guesses than unsatisfiable problems do; and these experiments strongly support that hypothesis: Of the 21 problems with max/min > 30, all but P0 are satisfiable, and all 32 of the problems with max/min < 1.7 are unsatisfiable. One might also expect the mean running time (the arithmetic average) to exceed the median running time, in problems like this—because bad luck can be significantly bad, though hopefully rare. Yet the mean is actually smaller than the median in 30 cases, about equally distributed between satisfiable and unsatisfiable.

The median is a nice measure because it is meaningful even in the presence of occasional timeouts. It’s also fair, because we are able to achieve the median time, or better, more often than not.

We should point out that input/output has been excluded from these time comparisons. Each satisfiability problem is supposed to appear within a computer’s memory as a simple list of clauses, after which the counting of mems actually begins. We include the cost of initializing the data structures and solving the problem, but then we stop counting before actually outputting a solution.

Some of the test cases in Table 6 and Fig. 96 represent different encodings of the same problem. For example, problems K0–K8 all demonstrate that the 8 × 8 queen graph can’t be colored with 8 colors. Similarly, problems Q0–Q9 all show that 9 colors will suffice. We’ve already discussed these examples above when considering alternative encodings; and we noted that the best solutions, K6 and Q5, are obtained with an extended order encoding and with Algorithm C. Therefore the fact that Algorithm L beats Algorithm C on problems K0, K1, K2, and K3 is somewhat irrelevant; those problems won’t occur in practice.

Problems L5 and L6 compare different ways to handle the at-most-one constraint. L6 is slightly better for Algorithm L, but Algorithm C prefers L5. Similarly, M1 and M2 compare different ways to deal with a more general cardinality constraint. Here M2 turns out to be better, although both are quite easy for Algorithm C and difficult for Algorithm L.

Fig. 97. Nine random running times of Algorithm C, sorted by their medians. (Unsatisfiable cases have solid dots or squares; satisfiable cases are hollow.)

We’ve already noted that Algorithm L shines with respect to random problems such as R1 and R2, and it dominates all competitors even more when unsatisfiable random 3SAT problems get even bigger. Lookahead methods are also successful in waerden problems like W1–W4.

Unsatisfiable Langford problems such as L3 and L4 are definitely bêtes noires for Algorithm C, although not so bad for Algorithm L. Even the world’s fastest CDCL solver, “Treengeling,” was unable to refute the clauses of langford(17) in 2013 until it had learned 26.7 billion clauses; this process took more than a week, using a cluster of 24 computers working together. By contrast, the backtrack method of exercise 7.2.2–21 was able to prove unsatisfiability after fewer than 4Tμ of computation—that’s about 50 minutes on a single vintage-2013 CPU.

We’ve now discussed every case where Algorithm L trounces Algorithm C, except for D5; and D5 is actually somewhat scandalous! It’s an inherently simple problem that hardware designers call a “miter”: Imagine two identical circuits that compute some function f(x_1,...,xn), one with gates g_1, ... , gm and another with corresponding gates g1′,…,gm′, all represented as in (24). The problem is to find x_1 ...xn for which the final results gm and gm′ aren’t equal. It’s obviously unsatisfiable. Furthermore, there’s an obvious way to refute it, by successively learning the clauses (g¯1∨g1′), (g¯1′∨g1), (g¯2∨g¯2′), (g¯2′∨g2), etc. In theory, therefore, Algorithm C will almost surely finish in polynomial time (see exercise 386). But in practice, the algorithm won’t discover those clauses without quite a lot of flailing around, unless special-purpose techniques are introduced to help it discover isomorphic gates.

Thus Algorithm C does have an Achilles heel or two. On the other hand, it is the clear method of choice in the vast majority of our test cases, and we can expect it to be the major workhorse for most of the satisfiability problems that we encounter in daily work. Therefore it behooves us to understand its behavior in some detail, not just to look at its total cost as measured in mems.

Table 7
ALGORITHM C’S EMPIRICAL BEHAVIOR ON THE HUNDRED TEST CASES

name

runtime

bytes

cells

nodes

learned

of size

triv

disc

sub

flushes

sat?

X2

0+2 Mμ

57 K

9 K

2K

1K

32.0 → 12.0

50%

6%

1%

30

U

K6

0+2 Mμ

314 K

46 K

1K

0K

15.8 → 11.8

22%

4%

3%

6

U

L5

1+1 Mμ

1841 K

210 K

0K

0 K

146.1 → 38.4

51%

23%

0%

0

S

P3

0+2 Mμ

96 K

19 K

2K

1K

18.4 → 12.6

4%

11%

1%

45

S

T1

0+6 Mμ

541 K

35 K

3K

1K

7.4 → 6.8

3%

2%

6%

9

S

T2

0+7 Mμ

574 K

37 K

4K

1K

7.2 → 6.8

1%

2%

4%

6

U

L6

0+8 Mμ

672 K

39 K

1K

0 K

195.9 → 67.8

86%

0%

0%

0

S

P0

0+11 Mμ

376 K

81 K

8K

4K

17.8 → 14.7

3%

10%

10%

28

U

K5

0+13 Mμ

294 K

55 K

3K

2K

18.6 → 12.4

33%

1%

1%

14

U

X1

0+13 Mμ

284 K

38 K

29 K

4K

6.3 → 5.8

0%

3%

8%

53

U

M4

0+24 Mμ

308 K

47 K

6K

4K

20.5 → 16.3

14%

2%

1%

3

S

S1

0+25 Mμ

366 K

72 K

9K

4K

34.0 → 26.7

22%

4%

1%

14

S

G4

0+29 Mμ

759 K

76 K

3K

2K

37.1 → 24.2

26%

0%

0%

1

S

N1

16+14 Mμ

19644 K

2314 K

41 K

0 K

629.3 → 291.7

44%

6%

0%

15

S

B1

0+31 Mμ

251 K

55 K

10 K

7K

13.5 → 11.3

3%

5%

4%

14

U

M2

0+32 Mμ

326 K

53 K

7K

5K

18.2 → 12.8

20%

1%

1%

6

U

L7

12+23 Mμ

14695 K

1758 K

2K

1 K

411.2 → 107.6

66%

4%

0%

0

S

E0

0+40 Mμ

571 K

95 K

5K

3K

30.2 → 19.3

14%

11%

0%

6

S

S4

1+69 Mμ

3291 K

600 K

6K

2K

17.2 → 12.6

19%

1%

1%

8

S

L8

1+72 Mμ

3047 K

224 K

3K

2 K

547.9 → 169.1

87%

0%

0%

0

S

M3

0+83 Mμ

493 K

84 K

13 K

9K

28.4 → 19.2

31%

0%

1%

1

S

Q2

0+87 Mμ

885 K

190 K

11 K

8K

61.7 → 45.8

36%

0%

0%

11

S

X6

0+93 Mμ

775 K

122 K

86 K

17 K

13.5 → 11.4

0%

3%

3%

32

U

F2

0+95 Mμ

714 K

118 K

42 K

22 K

14.3 → 13.1

0%

2%

4%

5

S

X4

1+98 Mμ

3560 K

158 K

24 K

3K

16.2 → 11.4

9%

2%

3%

623

S

X5

1+106 Mμ

3747 K

166 K

23 K

3K

16.5 → 11.0

11%

3%

3%

726

U

M1

0+131 Mμ

483 K

84 K

16 K

12 K

23.2 → 13.4

33%

1%

0%

1

U

Q5

0+143 Mμ

708 K

157 K

13 K

11 K

28.8 → 23.6

21%

2%

2%

6

S

L1

0+157 Mμ

597 K

139 K

21 K

18 K

36.7 → 19.0

60%

3%

0%

30

U

S2

0+176 Mμ

722 K

161 K

29 K

17 K

37.5 → 27.5

33%

3%

1%

8

U

S3

1+201 Mμ

2624 K

471 K

12 K

6K

14.5 → 9.8

21%

1%

2%

1

U

Q4

0+213 Mμ

781 K

175 K

19 K

16 K

29.2 → 23.3

25%

3%

1%

6

S

L2

0+216 Mμ

588 K

136 K

23 K

20 K

36.2 → 17.4

75%

1%

0%

6

U

X3

0+235 Mμ

1000 K

191 K

61 K

25 K

37.7 → 19.3

34%

1%

2%

14

U

G3

0+251 Mμ

1035 K

145 K

12 K

9K

57.9 → 28.1

42%

1%

0%

0

S

Q0

0+401 Mμ

1493 K

342 K

37 K

28 K

63.3 → 40.0

50%

0%

0%

14

S

Q1

0+464 Mμ

1516 K

343 K

41 K

33 K

63.0 → 41.0

45%

0%

0%

14

S

T4

0+546 Mμ

2716 K

544 K

202 K

18 K

218.3 → 61.5

83%

1%

0%

3018

S

Q9

0+555 Mμ

1409 K

343 K

152 K

71 K

26.7 → 20.6

3%

5%

2%

99

S

Q3

0+613 Mμ

1883 K

448 K

27 K

22 K

60.1 → 40.3

41%

1%

1%

7

S

W1

0+626 Mμ

848 K

208 K

71 K

63 K

20.8 → 13.4

5%

14%

1%

28

U

Q6

0+646 Mμ

1211 K

266 K

40 K

35 K

30.4 → 23.2

30%

1%

1%

2

S

M6

0+660 Mμ

1378 K

266 K

80 K

52 K

34.0 → 22.2

33%

1%

1%

59

S

B2

0+668 Mμ

906 K

216 K

96 K

75 K

17.1 → 13.2

4%

5%

2%

16

U

T6

1+668 Mμ

2355 K

291 K

34 K

25 K

41.4 → 19.1

57%

0%

1%

11

U

D4

0+669 Mμ

1009 K

186 K

35 K

28 K

55.7 → 15.9

70%

0%

0%

2

S

M5

0+677 Mμ

1183 K

219 K

73 K

48 K

32.6 → 20.2

37%

1%

1%

139

U

R1

0+756 Mμ

913 K

220 K

87 K

74 K

17.3 → 12.4

3%

8%

0%

9

S

F1

0+859 Mμ

1485 K

311 K

218 K

135 K

17.6 → 15.1

1%

3%

3%

6

U

O2

7+1069 Mμ

18951 K

3144 K

3K

2K

17.0 → 9.5

35%

0%

0%

1

S

Q8

0+1107 Mμ

1786 K

437 K

184 K

109 K

29.4 → 20.2

6%

6%

1%

109

S

C5

0+1127 Mμ

1987 K

419 K

159 K

104 K

24.4 → 16.5

12%

2%

1%

776

S

D2

0+1159 Mμ

962 K

177 K

54 K

45 K

51.8 → 11.5

73%

0%

0%

2

U

C3

0+1578 Mμ

2375 K

571 K

190 K

96 K

49.7 → 23.4

39%

3%

2%

11

S

D1

0+1707 Mμ

1172 K

230 K

76 K

62 K

45.1 → 11.6

73%

0%

0%

2

U

T5

1+1735 Mμ

3658 K

617 K

80 K

59 K

72.5 → 40.9

50%

0%

0%

43

S

Q7

0+1761 Mμ

2055 K

419 K

515 K

118 K

33.9 → 20.3

9%

7%

0%

12

S

D3

0+1807 Mμ

1283 K

254 K

77 K

64 K

57.3 → 14.0

80%

0%

0%

1

S

R2

0+1886 Mμ

1220 K

296 K

173 K

149 K

17.0 → 11.8

3%

9%

0%

14

U

O1

7+2212 Mμ

18928 K

3140 K

5K

3K

17.3 → 8.9

39%

0%

0%

4

U

W3

0+2422 Mμ

1819 K

448 K

191 K

174 K

19.3 → 15.5

2%

12%

1%

18

S

P2

0+2435 Mμ

2039 K

504 K

378 K

301 K

20.9 → 13.7

3%

11%

1%

45

U

C6

0+2792 Mμ

2551 K

560 K

305 K

217 K

27.0 → 17.0

20%

2%

1%

492

U

E1

0+2902 Mμ

2116 K

453 K

180 K

144 K

38.0 → 20.5

21%

18%

0%

2

S

P1

0+3280 Mμ

2726 K

674 K

819 K

549 K

18.2 → 14.4

0%

9%

3%

45

U

G6

1+3941 Mμ

3523 K

647 K

380 K

253 K

31.0 → 17.8

31%

0%

0%

0

S

C9

13+4220 Mμ

35486 K

4923 K

116 K

32 K

11.8 → 9.9

5%

1%

1%

4986

S

C2

0+4625 Mμ

2942 K

712 K

442 K

255 K

46.1 → 18.8

42%

4%

1%

15

U

K4

0+5122 Mμ

1858 K

446 K

267 K

241 K

19.6 → 13.7

19%

2%

1%

5

U

C1

0+5178 Mμ

2532 K

613 K

510 K

311 K

48.9 → 17.0

48%

6%

1%

20

U

G7

1+6070 Mμ

4227 K

771 K

546 K

369 K

32.5 → 17.6

35%

0%

0%

0

U

C8

13+6081 Mμ

35014 K

4823 K

151 K

58 K

15.3 → 10.7

15%

1%

1%

8067

U

T7

1+6467 Mμ

5428 K

544 K

333 K

108 K

26.8 → 15.3

32%

1%

1%

14565

S

C7

8+7029 Mμ

20971 K

3174 K

908 K

32 K

9.5 → 8.4

0%

3%

0%

4965

S

T8

1+7046 Mμ

5322 K

517 K

356 K

117 K

26.9 → 15.0

33%

0%

1%

15026

U

W2

0+7785 Mμ

3561 K

884 K

501 K

432 K

34.7 → 21.3

13%

17%

1%

28

S

G5

1+7799 Mμ

4312 K

844 K

642 K

446 K

33.4 → 17.4

39%

0%

0%

0

U

G1

0+8681 Mμ

5052 K

1221 K

631 K

350 K

61.1 → 34.1

38%

1%

2%

55

S

K1

0+9813 Mμ

2864 K

685 K

405 K

360 K

36.2 → 18.4

53%

2%

0%

13

U

X7

1+11857 Mμ

6235 K

697 K

1955 K

224 K

40.6 → 23.7

35%

0%

1%

31174

S

K0

0+11997 Mμ

3034 K

731 K

493 K

421 K

35.6 → 19.4

45%

2%

0%

14

U

K2

0+12601 Mμ

3028 K

729 K

500 K

427 K

34.8 → 18.0

46%

2%

0%

12

U

A2

0+13947 Mμ

3766 K

843 K

645 K

585 K

34.4 → 15.9

32%

1%

0%

0

S

K8

0+15033 Mμ

2748 K

680 K

821 K

699 K

21.2 → 13.1

8%

15%

1%

93

U

P4

0+16907 Mμ

6936 K

1721 K

1676 K

1314 K

36.5 → 24.0

5%

11%

1%

33

S

A1

0+17073 Mμ

3647 K

815 K

763 K

701 K

30.7 → 14.7

29%

2%

0%

0

U

T3

0+19266 Mμ

10034 K

2373 K

2663 K

323 K

291.8 → 72.9

86%

1%

0%

34265

U

K7

0+20577 Mμ

3168 K

721 K

1286 K

828 K

23.3 → 13.5

9%

15%

0%

9

U

K3

0+20990 Mμ

3593 K

878 K

453 K

407 K

36.7 → 19.0

55%

2%

0%

6

U

W4

0+21295 Mμ

3362 K

834 K

977 K

899 K

19.0 → 14.1

4%

15%

0%

21

U

M8

0+22281 Mμ

4105 K

994 K

992 K

785 K

37.3 → 20.5

43%

1%

1%

6

U

G2

0+23424 Mμ

6910 K

1685 K

1198 K

701 K

68.8 → 34.3

47%

1%

1%

120

U

D5

0+24141 Mμ

3232 K

779 K

787 K

654 K

63.5 → 13.4

78%

0%

0%

2

U

M7

0+24435 Mμ

4438 K

1077 K

1047 K

819 K

40.6 → 23.3

42%

1%

1%

6

S

C4

1+31898 Mμ

8541 K

2108 K

1883 K

1148 K

60.6 → 25.7

42%

4%

1%

12

S

G8

7+35174 Mμ

24854 K

2992 K

4350 K

1101 K

48.0 → 34.7

9%

0%

0%

1523

S

E2

0+53739 Mμ

5454 K

1258 K

2020 K

1658 K

41.5 → 20.8

25%

21%

0%

3

S

X8

2+248789 Mμ

12814 K

2311 K

17005 K

3145 K

56.4 → 22.5

63%

0%

0%

330557

U

L3

0+295571 Mμ

19653 K

4894 K

7402 K

6886 K

70.7 → 31.0

63%

8%

0%

30

U

L4

0+677815 Mμ

22733 K

5664 K

8545 K

7931 K

78.6 → 35.4

86%

0%

0%

5

U

Table 7 summarizes the salient statistics, again listing all cases in order of their median running time (exclusive of input and output). Each running time is actually broken into two parts, ‘x+y’, where x is the time to initialize the data structures in step C1 and y is the time for the other steps, both rounded to megamems. For example, the exact median processing time for case L5 was 1,484,489μ to initialize, then 655,728μ to find a solution; this is shown as ‘1+1 Mμ’ in the third line of the table. The time for initialization is usually negligible except when there are many clauses, as in problem N1.

The median run of problem L5 also allocated 1,841,372 bytes of memory for data; this total includes the space needed for 210,361 cells in the MEM array, at 4 bytes per cell, together with other arrays such as VAL, OVAL, HEAP, etc. The implementation considered here keeps unlearned binary clauses in a separate BIMP table, as explained in the answer to exercise 267.

This run of L5 found a solution after implicitly traversing a search tree with 138 “nodes.” The number of nodes, or “decisions,” is the number of times step C6 of the algorithm goes to step C3. It is shown as ‘0 K’ in Table 7, because the node counts, byte counts, and cell counts are rounded to the nearest thousand.

The number of nodes always exceeds or equals the number of learned clauses, which is the number of conflicts detected at levels d > 0. (See step C7.) In the case of problem L5, only 84 clauses were learned; so again the table reports ‘0 K’. These 84 clauses had average length r + 1 = 146.1; then the simplification process of exercise 257 reduced this average to just 38.4. Nevertheless, the resulting simplified clauses were still sufficiently long that the “trivial” clauses discussed in exercise 269 were sometimes used instead; this substitution happened 43 times (51 %). Furthermore 19 of the learned clauses (23 %) were immediately discarded, using the method of exercise 271. These percentages show up in the ‘triv’ and ‘disc’ columns of the table.

Sometimes, as in problems D1–D5, a large majority of the learned clauses were replaced by trivial ones; on the other hand, 27 of the 100 cases turned out to be less than 10% trivial in this sense. Table 7 also shows that the discard rate was 5% or more in 26 cases. The ‘sub’ column refers to learned clauses that were “subsumed on the fly” by the technique of exercise 270; this optimization is less common, yet it occurs often enough to be worthwhile.

The great variety in our examples is reflected in the variety of behaviors exhibited in Table 7, although several interesting trends can also be perceived. For example, the number of nodes is naturally correlated with the number of learned clauses, and both statistics tend to grow as the total running time increases. But there are significant exceptions: Two outliers, O1 and O2, have a remarkably high ratio of mems per learned clause, because of their voluminous data.

The penultimate column of Table 7 counts how often Algorithm C decided to restart itself after flushing unproductive literals from its current trail. This quantity does not simply represent the number of times step C5 discovers that M ≥ Mf ; it depends also on the current agility level (see (127)) and on the parameter ψ in Table 4. Some problems, like A1 and A2, had such high agility that they were solved satisfactorily with no restarts whatsoever; but another one, T4, finished in about 500 megamems after restarting more than 3000 times.

The number of “purges” (recycling phases) is not shown, but it can be estimated from the number of learned clauses (see exercise 508). An aggressive purging policy has kept the total number of memory cells comfortably small.

Tuning up the parameters

Table 7 shows that the hardest problem of all for Algorithm C in these experiments, L4, found itself substituting trivial clauses 86% of the time but making only 5 restarts. That test case would probably have been solved much more quickly if the algorithm’s parameters had been specially adjusted for instances of the Langford problem.

Algorithm C, as implemented in the experiments above, has ten major parameters that can be modified by the user on each run:

α, tradeoff between p and q in clause RANGE scores (see Eq. (123));
ρ, damping factor in variable ACT scores (see after (118));
ϱ, damping factor in clause ACT scores (see Eq. (125));

Δp, initial value of the purging threshold Mp (see after (125));
δp, amount of gradual increase in Mp (see after (125));
τ, threshold used to prefer trivial clauses (see answer to exercise 269);
w, full “warmup” runs done after a restart (see answer to exercise 287);
p, probability of choosing a decision variable at random (see exercise 266);

P, probability that OVAL(k) is initially even;
ψ, agility threshold for flushing (see Table 4).

The values for these parameters initially came from seat-of-the-pants guesses
α=0.2,ρ=0.95,   ϱ=0.999,  Δp=20000,   δP=500,               τ=1,  w=0,  p=0.02,  P=0,  ψ=0.166667;(193)

and those defaults gave reasonably good results, so they were used happily for many months (although there was no good reason to believe that they couldn’t be improved). Then finally, after the author had assembled the set of 100 test cases in Table 6, it was time to decide whether to recommend the default values (193) or to come up with a better set of numbers.

Parameter optimization for general broad-spectrum use is a daunting task, not only because of significant differences between species of SAT instances but also because of the variability due to random choices when solving any specific instance. It’s hard to know whether a change of parameter will be beneficial or harmful, when running times are so highly erratic. Ouch—Fig. 97 illustrates dramatic variations even when all ten parameters are held fixed, and only the seed for random numbers is changed! Furthermore the ten parameters are not at all independent: An increase in ρ, say, might be a good thing, but only if the other nine parameters are also modified appropriately. How then could any set of defaults be recommended, without an enormous expense of time and money?

Fortunately there’s a way out of this dilemma, thanks to advances in the theory of learning. F. Hutter, H. H. Hoos, K. Leyton-Brown, and T. Stützle have developed a tool called ParamILS intended specifically for making such tuneups [J. Artificial Intelligence Research 36 (2009), 267–306]; the ‘ILS’ in this name stands for “iterated local search.” The basic idea is to start with a representative training set of not-too-hard problems, and to carry out random walks in the 10-dimensional parameter space using sophisticated refinements of WalkSAT-like principles. The best parameters discovered during this training session are then evaluated on more difficult problems outside the training set.

In March 2015, Holger Hoos helped the author to tune Algorithm C using ParamILS. The resulting parameters then yielded Fig. 97, and Table 7, and many other runtime values discussed above and below. Our training set consisted of 17 problems that usually cost less than 200 Mμ with the original parameters (193), namely {K5, K6, M2, M4, N1, S1, S4, X4, X6} together with stripped-down versions of {A1, C2, C3, D1, D2, D3, D4, K0}. For example, instead of the vector x_1 ... x_100 required by problem A1, we looked only for a shorter vector x = x_1 ... x_62, now with νx = 20; instead of D1 and D2 we sought 13-bit factors of 31415926; instead of K0 we tried to 9-color the SGB graph jean.

Ten independent training runs with ParamILS gave ten potential parameter settings (αi, ρi,...,ψi). We evaluated them on our original 17 benchmarks, together with 25 others that were a bit more difficult: {F1, F2, S2, S3, T4, X5}, plus less-stripped-down variants of {A1, A2, A2, C7, C7, D3, D4, F1, F2, G1, G1, G2, G2, G8, K0, O1, O2, Q0, Q2}. For each of the ten shortlisted parameter settings, we ran each of these 17 + 25 problems with each of the random seeds {1, 2,..., 25}. Finally, hurrah, we had a winner: The parameters (α, ρ,...,ψ) with minimum total running time in this experiment were
α=0.4,ρ=0.9,ϱ=0.9995,Δp=1000,δp=500,τ=10,w=0,p=0.02,P=0.5,ψ=0.05.(194)

And these are now the recommended defaults for general-purpose use.

How much have we thereby gained? Figure 98 compares the running times of our 100 examples, before and after tuning. It shows that the vast majority—77 of them—now run faster; these are the cases to the right of the dotted line from (1 Mμ, 1 Mμ) to (1 Tμ, 1 Tμ). Half of the cases experience a speedup exceeding 1.455; 27 of them now run more than twice as fast as they previously did.

Of course every rule has exceptions. The behavior of case P4 has gotten spectacularly worse, almost three orders of magnitude slower! Indeed, we saw earlier in Fig. 97 that this case has an amazingly unstable running time; further peculiarities of P4 are discussed in exercise 511.

Our other major SAT solver, Algorithm L, also has parameters, notably
α, magic tradeoff coefficient in heuristic scores (see Eq. (64));
β, damping factor for double-look triggering (see step Y1);
γ, clause weight per literal in heuristic scores (see exercise 175);
ε, offset in heuristic scores (see answer to exercise 146);

Θ, maximum heuristic score threshold (see answer to exercise 145);

Y, maximum depth of double-lookahead (see step Y1).

ParamILS suggests the following default values, which have been used in Fig. 96:

α=3.5,β=0.9998,γ=0.2,ε=0.001,Θ=20.0,Y=1,(195)

Returning to Fig. 98, notice that the change from (193) to (194) has substantially hindered cases G3 and G4, which are examples of test pattern generation. Evidently such clauses have special characteristics that make them prefer special settings of the parameters. Our main reason for introducing parameters in the first place was, of course, to allow tweaking for different families of clauses.

Fig. 98. Median running times of Algorithm C, before and after its parameters were tuned.

Instead of finding values of (α, ρ,...,ψ) that give good results in a broad spectrum of applications, we can clearly use a system like ParamILS to find values that are specifically tailored to a particular class of problems. In fact, this task is easier. For example, Hoos and the author asked for settings of the ten parameters that will tend to make Algorithm C do its best on problems of the form waerden (3, k; n). A pair of ParamILS runs, based solely on the easy training cases waerden (3, 9; 77) and waerden (3, 10; 95), suggested the parameters
α=0.5,ρ=0.9995,ϱ=0.99,Δp=100,δp=10,τ=10,w=8,p=0.01,P=0.5,ψ=0.15,(196)

and this set indeed works very well. Figure 99 shows typical details, with 7 ≤ k ≤ 14 and with nine independent sample runs for every choice of k and n. Each unsatisfiable instance has n = W(3, k), as given in the table following (10) above; each satisfiable instance has n = W (3,k)−1. The fastest run using default parameters (194) has been paired in Fig. 99 with the fastest run using waerden-tuned parameters (196); similarly, the second-fastest, ... , second-slowest, and slowest runs have also been paired. Notice that satisfiable instances tend to take an unpredictable amount of time, as in Fig. 97. In spite of the fact that the new parameters (196) were found by a careful study of just two simple instances, they clearly yield substantial savings when applied to much, much harder problems of a similar nature. (See exercise 512 for another instructive example.)

Fig. 99. Running times of Algorithm C on clauses waerden (3, k; n), with and without special tuning.

Exploiting parallelism

Our focus in the present book is almost entirely on sequential algorithms, but we should be aware that the really tough instances of SAT are best solved by parallel methods.

Problems that are amenable to backtracking can readily be decomposed into subproblems that partition the space of solutions. For example, if we have 16 processors available, we can start them off on independent SAT instances in which variables x_1x_2x_3x_4 have been forced to equal 0000, 0001, ... , 1111.

A naïve decomposition of that kind is rarely the best strategy, however. Perhaps only one of those sixteen cases is really challenging. Perhaps some of the processors are slower than others. Perhaps several processors will learn new clauses that the other processors ought to know. Furthermore, the splitting into subproblems need not occur only at the root of the search tree. Careful load-balancing and sharing of information will do much better. These challenges were addressed by a pioneering system called PSATO [H. Zhang, M. P. Bonacina, and J. Hsiang, Journal of Symbolic Computation 21 (1996), 543–560].

A much simpler approach should also be mentioned: We can start up many different solvers, or many copies of the same solver, with different sources of random numbers. As soon as one has finished, we can then terminate the others.

The best parallelized SAT solvers currently available are based on the “cube and conquer” paradigm, which combines conflict driven clause learning with lookahead techniques that choose branch variables for partitioning; see M. J. H. Heule, O. Kullmann, S. Wieringa, and A. Biere, LNCS 7261 (2012), 50–65. In particular, this approach is excellent for the waerden problems.

Today has proved to be an epoch in my Logical work.

. . . I think of calling it the ‘Genealogical Method.’

— CHARLES L. DODGSON, Diary (16 July 1894)

The method of showing a statement to be tautologous consists merely of constructing a table under it in the usual way and observing that the column under the main connective is composed entirely of ‘T’s.

— W. V. O. QUINE, Mathematical Logic (1940)

A brief history

The classic syllogism “All men are mortal; Socrates is a man; hence Socrates is mortal” shows that the notion of resolution is quite ancient:

¬Man ∨ Mortal; ¬Socrates ∨ Man; ∴ ¬Socrates ∨ Mortal.

Of course, algebraic demonstrations that (¬x∨y)∧(¬z∨x) implies (¬z∨y), when x, y, and z are arbitrary Boolean expressions, had to wait until Boole and his 19th-century followers brought mathematics to bear on the subject. The most notable contributor, resolutionwise, was perhaps C. L. Dodgson, who spent the last years of his life working out theories of inference by which complex chains of reasoning could be analyzed by hand. He published Symbolic Logic, Part I, in 1896, addressing it to children and to the young-in-heart by using his famous pen name Lewis Carroll. Section VII.II.§3 of that book explains and illustrates how to eliminate variables by resolution, which he called the Method of Underscoring.

When Dodgson died unexpectedly at the beginning of 1898, his nearly complete manuscript for Symbolic Logic, Part II, vanished until W. W. Bartley III was able to resurrect it in 1977. Part II was found to contain surprisingly novel ideas—especially its Method of Trees, which would have completely changed the history of mechanical theorem proving if it had come to light earlier. In this method, which Carroll documented at length in a remarkably clear and entertaining way, he constructed search trees essentially like Fig. 82, then converted them into proofs by resolution. Instead of backtracking as in Algorithm D, which is a recursive depth-first method, he worked breadth-first: Starting at the root, he exploited unit clauses when possible, and branched on binary (or even ternary) clauses when necessary, successively filling out all unfinished branches level-by-level in hopes of being able to reuse computations.

Logicians of the 20th century took a different tack. They basically dealt with the satisfiability problem in its equivalent dual form as the tautology problem, namely to decide when a Boolean formula is always true. But they dismissed tautology-checking as a triviality, because it could always be solved in a finite number of steps by just looking at the truth table. Logicians were far more interested in problems that were provably unsolvable in finite time, such as the halting problem—the question of whether or not an algorithm terminates. Nobody was bothered by the fact that an n-variable function has a truth table of length 2n, which exceeds the size of the universe even when n is rather small.

Practical computations with disjunctive normal forms were pioneered by Archie Blake in 1937, who introduced the “consensus” of two implicants, which is dual to the resolvent of two clauses. Blake’s work was, however, soon forgotten; E. W. Samson, B. E. Mills, and (independently) W. V. O. Quine rediscovered the consensus operation in the 1950s, as discussed in exercise 7.1.1–31.

The next important step was taken by E. W. Samson and R. K. Mueller [Report AFCRC-TR-55-118 (Cambridge, Mass.: Air Force Cambridge Research Center, 1955), 16 pages], who presented an algorithm for the tautology problem that uses consensus to eliminate variables one by one. Their algorithm therefore was equivalent to SAT solving by successively eliminating variables via resolution. Samson and Mueller demonstrated their algorithm by applying it to the unsatisfiable clauses that we considered in (112) above.

Independently, Martin Davis and Hilary Putnam had begun to work on the satisfiability problem, motivated by the search for algorithms to deduce formulas in first-order logic—unlike Samson, Mills, and Mueller, who were chiefly interested in synthesizing efficient circuits. Davis and Putnam wrote an unpublished 62-page report “Feasible computational methods in the propositional calculus” (Rensselaer Polytechnic Institute, October 1958) in which a variety of different approaches were considered, such as the removal of unit clauses and pure literals, as well as “case analysis,” that is, backtracking with respect to the subproblems F |x and F|x¯. As an alternative to case analysis, they also discussed eliminating the variable x by resolution. The account of this work that was eventually published [JACM 7 (1960), 201–215] concentrated on hand calculation, and omitted case analysis in favor of resolution; but when the process was later implemented on a computer, jointly with George Logemann and Donald Loveland [CACM 5 (1962), 394–397], the method of backtracking through different cases was found to work better with respect to memory requirements. (See Davis’s account of these developments in Handbook of Automated Reasoning (2001), 3–15.)

This early work didn’t actually cause the satisfiability problem to appear on many people’s mental radar screens, however. Far from it; ten years went by before SAT became an important buzzword. The picture changed in 1971, when Stephen A. Cook showed that satisfiability is the key to solving NP-complete problems: He proved that any algorithm to solve a decision problem in nondeterministic polynomial time can be represented efficiently as a conjunction of ternary clauses to be satisfied. (See STOC 3 (1971), 151–158. We’ll study NP-completeness in Section 7.9.) Thus, a great multitude of hugely important problems could all be solved rather quickly, if we could only devise a decent algorithm for a single problem, 3SAT; and 3SAT seemed almost absurdly simple to solve.

A year of heady optimism following the publication of Cook’s paper soon gave way to the realization that, alas, 3SAT might not be so easy after all. Ideas that looked promising in small cases didn’t scale well, as the problem size was increased. Hence the central focus of work on satisfiability largely retreated into theoretical realms, unrelated to programming practice, except for occasional studies that used SAT as a simple model for the behavior of backtracking algorithms in general. Examples of such investigations, pioneered by A. T. Goldberg, P. W. Purdom, Jr., C. A. Brown, J. V. Franco, and others, appear in exercises 213–216. See P. W. Purdom, Jr., and G. N. Haven, SICOMP 26 (1997), 456–483, for a survey of subsequent progress on questions of that kind.

The state of SAT art in the early 90s was well represented by an international programming competition held in 1992 [see M. Buro and H. Kleine Büning, Bulletin EATCS 49 (February 1993), 143–151]. The winning programs in that contest can be regarded as the first successful lookahead solvers on the path from Algorithm A to Algorithm L. Max Böhm “took the gold” by choosing the next branch variable based on lexicographically maximal (H_1(x),...,Hn(x)), where

Hk(x)=hk(x)+hk(x¯)+min(hk(x),hk(x¯)),hk⁢(x)=|{C∈F|x∈C,|C|=k}|.

[See M. Böhm and E. Speckenmeyer, Ann. Math. Artif. Intelligence 17 (1996), 381–400. A. Rauzy had independently proposed a somewhat similar branching criterion in 1988; see Revue d’intelligence artificielle 2 (1988), 41–60.] The silver medal went to Hermann Stamm, who used strong components of the dependency digraph to narrow the search at each branch node.

Advances in practical algorithms for satisfiability now began to take off. The benchmark problems of 1992 had been chosen at random, but the DIMACS Implementation Challenge of 1993 featured also a large number of structured instances of SAT. The main purpose of this “challenge” was not to crown a winner, but to bring more than 100 researchers together for a three-day workshop, at which they could compare and share results. In retrospect, the best overall performance at that time was arguably achieved by an elaborate lookahead solver called C-SAT, which introduced techniques for detailed exploration of the first-order effects of candidate literals [see O. Dubois, P. Andre, Y. Boufkhad, and J. Carlier, DIMACS 26 (1996), 415–436]. Further refinements leading towards the ideas in Algorithm L appeared in a Ph.D. thesis by Jon W. Freeman (Univ. of Pennsylvania, 1995), and in the work of Chu Min Li, who introduced double lookahead [see Information Processing Letters 71 (1999), 75–80]. The weighted binary heuristic (67) was proposed by O. Dubois and G. Dequen, Proc. International Joint Conference on Artificial Intelligence 17 (2001), 248–253.

Meanwhile the ideas underlying Algorithm C began to emerge. Matthew L. Ginsberg [J. Artificial Intelligence Research 1 (1993), 25–46] showed that efficient backjumping was possible while remembering only at most two learned clauses for each variable. João P. Marques-Silva, in his 1995 thesis directed by Karem A. Sakallah, discovered how to turn unit-propagation conflicts into one or more clauses learned at “unique implication points,” thus enhancing the potential for backjumping past decisions that didn’t affect the conflict. [See IEEE Trans. C48 (1999), 506–521.] Similar methods were developed independently by Roberto J. Bayardo, Jr., and Robert C. Schrag [AAAI Conf. 14 (1997), 203–208], who considered only the special case of new clauses that include the current decision literal, but introduced techniques for purging a learned clause when one of its literals was forced to flip its value. These new methods gave significant speedups on benchmark problems related to industrial applications.

The existence of fast SAT solvers, coupled with Gunnar Stålmarck’s new ideas about applying logic to computer design [see Swedish patent 467076 (1992)], led to the introduction of bounded model checking techniques by Armin Biere, Alessandro Cimatti, Edmund Clarke, and Yunshan Zhu [LNCS 1579 (1999), 193–207]. Satisfiability techniques had also been introduced to solve classical planning problems in artificial intelligence [Henry Kautz and Bart Selman, Proc. European Conf. Artificial Intelligence 10 (1992), 359–363]. Designers could now verify much larger models than had been possible with BDD methods.

The major breakthroughs appeared in a solver called Chaff [M. W. Moskewicz, C. F. Madigan, Y. Zhao, L. Zhang, and S. Malik, ACM/IEEE Design Automation Conf. 38 (2001), 530–535], which had two especially noteworthy innovations: (i) “VSIDS” (the Variable State Independent Decaying Sum heuristic), a surprisingly effective way to select decision literals, which also worked well with restarts, and which suggested the even better ACT heuristic of Algorithm C that soon replaced it; also (ii) lazy data structures with two watched literals per clause, which made unit propagation much faster with respect to large learned clauses. (A somewhat similar watching scheme, introduced earlier by H. Zhang and M. Stickel [J. Automated Reasoning 24 (2000), 277–296], had the disadvantage that it needed to be downdated while backtracking.)

These exciting developments sparked a revival of international SAT competitions, which have been held annually since 2002. The winner in 2002, BerkMin by E. Goldberg and Y. Novikov, has been described well in Discrete Applied Mathematics 155 (2007), 1549–1561. And year after year, these challenging contests have continued to spawn further progress. By 2010, more than twice as many benchmarks could be solved in a given period of time as in 2002, using the programs of 2002 and 2010 on the computers of 2010 [see M. Järvisalo, D. Le Berre, O. Roussel, and L. Simon, AI Magazine 33,1 (Spring 2012), 89–94].

The overall champion in 2007 was SATzilla, which was actually not a separate SAT solver but rather a program that knew how to choose intelligently between other solvers on any given instance. SATzilla would first take a few seconds to compute basic features of a problem: the distribution of literals per clause and clauses per literal, the balance between positive and negative occurrences of variables, the proximity to Horn clauses, etc. Samples could quickly be taken to estimate how many unit propagations occur at levels 1, 4, 16, 64, 256, and how many decisions are needed before reaching a conflict. Based on these numbers, and experience with the performance of the other solvers on the previous year’s benchmarks, SATzilla was trained to select the algorithm that appeared most likely to succeed. This “portfolio” approach, which tunes itself nicely to the characteristics of vastly different sets of clauses, has continued to dominate the international competitions ever since. Of course portfolio solvers rely on the existence of “real” solvers, invented independently and bug-free, which shine with respect to particular classes of problems. And of course the winner of competitions may not be the best actual system for practical use. [See L. Xu, F. Hutter, H. H. Hoos, and K. Leyton-Brown, J. Artificial Intelligence Research 32 (2008), 565–606; LNCS 7317 (2012), 228–241; CACM 57, 5 (May 2014), 98–107.]

Historical notes about details of the algorithms, and about important related techniques such as preprocessing and encoding, have already been discussed above as the algorithms and techniques were described.

One recurring theme appears to be that the behavior of SAT solvers is full of surprises: Some of the most important improvements have been introduced for what has turned out to be the wrong reasons, and a theoretical understanding is still far from adequate.

[The next future breakthrough might come from “variable learning,” as suggested by Tseytin’s idea of extended resolution: Just as clause learning increases the number of clauses, m, we might find good ways to increase the number of variables, n. The subject seems to be far from fully explored.]

Exercises

1. [10] What are the shortest (a) satisfiable (b) unsatisfiable sets of clauses?

2. [20] Travelers to the remote planet Pincus have reported that all the healthy natives like to dance, unless they’re lazy. The lazy nondancers are happy, and so are the healthy dancers. The happy nondancers are healthy; but natives who are lazy and healthy aren’t happy. Although the unhappy, unhealthy ones are always lazy, the lazy dancers are healthy. What can we conclude about Pincusians, based on these reports?

3. [M21] Exactly how many clauses are in waerden (j, k; n)?

4. [22] Show that the 32 constraints of waerden (3, 3; 9) in (9) remain unsatisfiable even if up to four of them are removed.

5. [M46] Is W(3, k) = Θ(k^2)?

▸ 6. [HM37] Use the Local Lemma to show that W(3, k) = Ω(k^2/(log k)3).

7. [21] Can one satisfy the clauses {(xi∨xi+2d∨xi+2d+1)|1≤i≤n−2d+1,d≥0}∪{(x¯i∨x¯i+2d∨x¯i+2d+1)|1≤i≤n−2d+1,d≥0}?

▸ 8. [20] Define clauses waerden(k_0, k_1,...,kb−1; n) that are satisfiable if and only if n < W (k_0, k_1,...,kb−1).

9. [24] Determine the value of W(2, 2, k) for all k ≥ 0. Hint: Consider k mod 6.

▸ 10. [21] Show that every satisfiability problem with m clauses and n variables can be transformed into an equivalent monotonic problem with m+n clauses and 2n variables, in which the first m clauses have only negative literals, and the last n clauses are binary with two positive literals.

11. [27] (M. Tsimelzon, 1994.) Show that a general 3SAT problem with clauses {C_1,...,Cm} and variables {1,...,n} can be reduced to a 3D matching problem of size 10m that involves the following cleverly designed triples:

Each clause Cj corresponds to 3×10 vertices, namely lj, l¯j, |l|j′, and |l|j″ for each l ∈ Cj, together with wj, xj, yj, and zj, and also j′k and j″k for 1 ≤ k ≤ 7. If i or ι¯ occurs in t clauses Cj_1, ... , Cjt, there are t “true” triples {ijk, ijk′, ijk″} and t “false” triples {ι¯jk, ijk′, ij1+(k⁢ mod t)″}, for 1 ≤ k ≤ t. Each clause Cj = (l_1 ∨ l_2 ∨ l_3) also spawns three “satisfiability” triples {l¯1j,j′1,j″1}, {l¯2j,j′1,j″2}, {l¯3j,j′1,j″3}; six “filler” triples {l_1j, j′2, j″1}, {l¯1j,j′3,j″1}, {l_2j, j′4, j″2}, {l¯2j,j′5,j″2}, {l_3j, j′6, j″3}, {l¯3j,j′7,j″3}; and twelve “gadget” triples {wj, j′2, j″4}, {wj, j′4, j″4}, {wj, j′6, j″4}, {xj, j′2, j″5}, {xj, j′5, j″5}, {xj, j′7, j″5}, {yj, j′3, j″6}, {yj, j′4, j″6}, {yj, j′7, j″6}, {zj, j′3, j″7}, {zj, j′5, j″7}, {zj, j′6, j″7}. Thus there are 27m triples altogether.

For example, Rivest’s satisfiability problem (6) leads to a 3D matching problem with 216 triples on 240 vertices; the triples that involve vertices 18 and 1¯8 are {18, 18′, 18″}, {1¯8, 18′, 11″}, {1¯8, 8′1, 8″2}, {18, 8′4, 8″2}, {1¯8, 8′5, 8″2}.

12. [21] (M. J. H. Heule.) Simplify (13) by exploiting the identity

S≤1(y1,…,yp)=∃t(S≤1(y1,…yj,t)∧S≤1(t¯,yj+1,…,yp)).

13. [24] Exercise 7.2.2.1–15 defines an exact cover problem that corresponds to Langford pairs of order n.

1. What are the constraints analogous to (12) when n = 4?

2. Show that there’s a simple way to avoid duplicate binary clauses such as those in (14), whenever an exact cover problem is converted to clauses using (13).

3. Describe the corresponding clauses langford (4) and langford′ (4).

14. [22] Explain why the clauses (17) might help a SAT solver to color a graph.

15. [24] By comparing the McGregor graph of order 10 in Fig. 76 with the McGregor graph of order 3 shown here, give a precise definition of the vertices and edges of the McGregor graph whose order is a given number n ≥ 3. Exactly how many vertices and edges are present in this graph, as a function of n?

16. [21] Do McGregor graphs have cliques of size 4?

17. [26] Let f(n) and g(n) be the smallest and largest values of r such that McGregor’s graph of order n can be 4-colored, and such that some color appears exactly r times. Use a SAT solver to find as many values of f(n) and g(n) as you can.

▸ 18. [28] By examining the colorings found in exercise 17, define an explicit way to 4-color a McGregor graph of arbitrary order n, in such a way that one of the colors is used at most 56n times. Hint: The construction depends on the value of n mod 6.

▸ 19. [29] Continuing exercise 17, let h(n) be the largest number of regions that can be given two colors simultaneously (without using the clauses (17)). Investigate h(n).

20. [40] In exactly how many ways can McGregor’s map (Fig. 76) be four-colored?

21. [22] Use a SAT solver to find a minimum-size kernel in the graph of Fig. 76.

22. [20] Color the graph C5⊠C5¯ with the fewest colors. (Two vertices of this graph can receive the same color if and only if they are a king move apart in a 5 × 5 torus.)

23. [20] Compare the clauses (18) and (19) to (20) and (21) in the case n = 7, r = 4.

▸ 24. [M34] The clauses obtained from (20) and (21) in the previous exercise can be simplified, because we can remove the two that contain the pure literal b12.

1. Prove that the literal b12 is always pure in (20) and (21), when r > n/2.

2. Show that b12 might also be pure in some cases when r < n/2.

3. The clauses obtained from (20) and (21) have many pure literals bjk when r has its maximum value n − 1. Furthermore, their removal makes other literals pure. How many clauses will remain in this case after all pure literals have been eliminated?

4. Show that the complete binary tree with n ≥ 2 leaves is obtained from complete binary trees with n′ and n″ = n − n′ leaves, where either n′ or n″ is a power of 2.

5. Let a(n, r) and c(n, r) be respectively the number of auxiliary variables bjk and the total number of clauses that remain after all of the pure auxiliary literals have been removed from (20) and (21). What are a(2k, 2k−1) and c(2k, 2k−1)?

6. Prove that a(n, r) = a(n, n″) = a(n, n′) for n″ ≤ r ≤ n′, and this common value is max1≤r<n a(n, r). Also a(n, r) = a(n, n − r); and c(n, r) ≥ c(n, n − r) if r ≤ n/2.

25. [21] Show that (18)–(19) and (20)–(21) are equally effective when r = 2.

26. [22] Prove that Sinz’s clauses (18) and (19) enforce the cardinality constraint x_1 + ··· + xn ≤ r. Hint: Show that they imply sjk=1 whenever x_1 + ··· + xj+k−1 ≥ k.

27. [20] Similarly, prove the correctness of Bailleux and Boufkhad’s (20) and (21). Hint: They imply bjk=1 whenever the leaves below node k contain j or more 1s.

▸ 28. [20] What clauses result from (18) and (19) when we want to ensure that x_1 + ··· + xn ≥ 1? (This special case converts arbitrary clauses into 3SAT clauses.)

▸ 29. [20] Instead of the single constraint x_1 + ··· + xn ≤ r, suppose we wish to impose a sequence of constraints x_1 + ··· + xi ≤ ri for 1 ≤ i ≤ n. Can this be done nicely with additional clauses and auxiliary variables?

30. [22] If auxiliary variables sjk are used as in (18) and (19) to make x_1 +···+ xn ≤ r, while sj′k are used to make x¯1+⋅⋅⋅+x¯n≤n−r, show that we may unify them by taking sk′j=sjk¯, for 1 ≤ j ≤ n − r, 1 ≤ k ≤ r. Can (20) and (21) be similarly unified?

▸ 31. [28] Let Ft(r) be the smallest n for which there is a bit vector x_1 ... xn with x_1 + ··· + xn = r and with no t equally spaced 1s. For example, F_3(12) = 30 because of the unique solution 101100011010000000010110001101. Discuss how Ft(r) might be computed efficiently with the help of a SAT solver.

32. [15] A list coloring is a graph coloring in which v’s color belongs to a given set L(v), for each vertex v. Represent list coloring as a SAT problem.

33. [21] A double coloring of a graph is an assignment of two distinct colors to every vertex in such a way that neighboring vertices share no common colors. Similarly, a q-tuple coloring assigns q distinct colors to each vertex. Find double and triple colorings of the cycle graphs C_5, C_7, C_9, ... , using as few colors as possible.

34. [HM26] The fractional coloring number χ*(G) of a graph G is defined to be the minimum ratio p/q for which G has a q-tuple coloring that uses p colors.

1. Prove that χ*(G) ≤ χ(G), and show that equality holds in McGregor’s graphs.

2. Let S_1, ... , SN be all the independent subsets of G’s vertices. Show that
χ*(G)=minλ1,…,λN≥0{λ1+⋅⋅⋅+λN|Σj=1N λj[v∈Sj]=1 for all vertices v}.

(This is a fractional exact cover problem.)

3. What is the fractional coloring number χ*(Cn) of the cycle graph Cn?

4. Consider the following greedy algorithm for coloring G: Set k ← 0 and G_0 ← G; while Gk is nonempty, set k ← k + 1 and Gk ← Gk−1 \ Ck, where Ck is a maximum independent set of Gk−1. Prove that k ≤ Hα(G)χ*(G), where α(G) is the size of G’s largest independent set; hence χ(G)/χ*(G) ≤ Hα(G) = O(log n). Hint: Let tv = 1/|Ci| if v ∈ Ci, and show that ∑v∈S tv ≤ H|S| whenever S is an independent set.

35. [22] Determine χ*(G) when G is (a) the graph of the contiguous United States (see 7.2.2.1–(113)); (b) the graph of exercise 22.

▸ 36. [22] A radio coloring of a graph, also known as an L(2, 1) labeling, is an assignment of integer colors to vertices so that the colors of u and v differ by at least 2 when u — v, and by at least 1 when u and v have a common neighbor. (This notion, introduced by Fred Roberts in 1988, was motivated by the problem of assigning channels to radio transmitters, without interference from “close” transmitters and without strong interference from “very close” transmitters.) Find a radio coloring of McGregor’s graph, Fig. 76, that uses only 16 consecutive colors.

37. [20] Find an optimum radio coloring of the contiguous USA graph, 7.2.2.1–(113).

38. [M25] How many consecutive colors are needed for a radio coloring of (a) the n × n square grid Pn ⎕ Pn? (b) the vertices {(x, y, z) | x, y, z ≥ 0, x + y + z = n}, which form a triangular grid with n + 1 vertices on each side?

39. [M46] Find an optimum radio coloring of the n-cube, for some value of n > 6.

40. [01] Is the factorization problem (22) unsatisfiable whenever z is a prime number?

41. [M21] Determine the number of Boolean operations ∧, ∨, ⊕ needed to multiply m-bit numbers by n-bit numbers with Dadda’s scheme, when 2 ≤ m ≤ n.

42. [21] Tseytin encoding analogous to (24) can be devised also for ternary operations, without introducing any additional variables besides those of the function being encoded. Illustrate this principle by encoding the basic operations x ← t ⊕ u ⊕ v and y ← 〈tuv〉 of a full adder directly, instead of composing them from ⊕, ∧, and ∨.

▸ 43. [21] For which integers n ≥ 2 do there exist odd palindromic binary numbers x = (xn ... x_1)2 = (x_1 ... xn)2 and y = (yn ... y_1)2 = (y_1 ... yn)2 such that their product xy = (z2n ... z_1)2 = (z_1 ... z2n)2 is also palindromic?

▸ 44. [30] (Maximum ones.) Find the largest possible value of νx + νy + ν(xy), namely the greatest total number of 1 bits, over all multiplications of 32-bit binary x and y.

45. [20] Specify clauses that constrain (zt ... z_1)2 to be a perfect square.

46. [30] Find the largest perfect square less than 2^100 that is a binary palindrome.

▸ 47. [20] Suppose a circuit such as Fig. 77 has m outputs and n inputs, with g gates that transform two signals into one and h gates that transform one signal into two. Find a relation between g and h, by expressing the total number of wires in two ways.

48. [20] The small circuit shown here has three inputs, three XOR gates, one fanout gate, eight wires, and one output. Which single-stuck-at faults are detected by each of the eight test patterns pqr?

49. [24] Write a program that determines exactly which of the 100 single-stuck-at faults of the circuit in Fig. 77 are detected by each of the 32 possible input patterns. Also find all the minimum sets of test patterns that will discover every such fault (unless it’s not detectable).

50. [24] Demonstrate Larrabee’s method of representing stuck-at faults by describing the clauses that characterize test patterns for the fault “x21 stuck at 1” in Fig. 77. (This is the wire that splits off of x_2 and feeds into x23 and x24, then to b_2 and b_3; see Table 1.)

51. [40] Study the behavior of SAT solvers on the problem of finding a small number of test patterns for all of the detectable single-stuck-at faults of the circuit prod (32, 32). Can a complete set of patterns for this large circuit be discovered “automatically” (without relying on number theory)?

52. [15] What clauses correspond to (29) and (30) when the second case on the left of Table 2, f(1, 0, 1, 0,..., 1) = 1, is taken into account?

▸ 53. [M20] The numbers in Table 2 are definitely nonrandom. Can you see why?

▸ 54. [23] Extend Table 2 using the rule in the previous exercise. How many rows are needed before f(x) has no M-term representation in DNF, when M = 3, 4, and 5?

55. [21] Find an equation analogous to (27) that is consistent with Table 2 and has every variable complemented. (Thus the resulting function is monotone decreasing.)

▸ 56. [22] Equation (27) exhibits a function matching Table 2 that depends on only 8 of the 20 variables. Use a SAT solver to show that we can actually find a suitable f that depends on only five of the xj.

▸ 57. [29] Combining the previous exercise with the methods of Section 7.1.2, exhibit a function f for Table 2 that can be evaluated with only six Boolean operations(!).

▸ 58. [20] Discuss adding the clauses p¯i,j∨q¯i,j to (29), (30), and (31).

59. [M20] Compute the exact probability that f^(x) in (32) differs from f(x) in (27).

60. [24] Experiment with the problem of learning f(x) in (27) from training sets of sizes 32 and 64. Use a SAT solver to find a conjectured function, f^(x); then use BDD methods to determine the probability that this f^(x) differs from f(x) for random x.

61. [20] Explain how to test when a set of clauses generated from a training set via (29)–(31) is satisfiable only by the function f(x) in (27).

62. [23] Try to learn a secret small-DNF function with N-bit training sets x(0), x(1), x(2), ... , where x(0) is random but each bit of x(k) ⊕ x(k−1) for k > 0 is 1 with probability p. (Thus, if p is small, successive data points will tend to be near each other.) Do such sets turn out to be more efficient in practice than the purely random ones that arise for p = 1/2?

▸ 63. [20] Given an n-network α = [i_1 : j_1][i_2 : j_2] ... [ir : jr], as defined in the exercises for Section 5.3.4, explain how to use a SAT solver to test whether or not α is a sorting network. Hint: Use Theorem 5.3.4Z.

64. [26] The exact minimum time T^(n) of a sorting network for n elements is a famous unsolved problem, and the fact that T^(9)=7 was first established in 1987 by running a highly optimized program for many hours on a Cray 2 supercomputer.

Show that this result can now be proved with a SAT solver in less than a second(!).

▸ 65. [28] Describe encodings of the Life transition function (35) into clauses.

1. Use only the variables xij′ and xij.

2. Use auxiliary variables as in the Bailleux and Boufkhad encoding (20)–(21), sharing intermediate results between neighboring cells as discussed in the text.

66. [24] Use a SAT solver to find short counterparts to Fig. 78 in which (a) X_1 = 𝗟𝗜𝗞𝗘; (b) X_2 = 𝗟𝗜𝗞𝗘. In each case X_0 should have the smallest possible number of live cells.

67. [24] Find a mobile chessboard path X_0 → X_1 → ... → X_21 with no more than five cells alive in each Xt. (The glider in (37) leaves the board after X_20.) How about X_22?

68. [39] Find a maximum-length mobile path in which 6 to 10 cells are always alive.

69. [23] Find all (a) still lifes and (b) oscillators of period > 1 that live in a 4×4 board.

70. [21] The live cells of an oscillator are divided into a rotor (those that change) and a stator (those that stay alive).

1. Show that the rotor cannot be just a single cell.

2. Find the smallest example of an oscillator whose rotor is .

3. Similarly, find the smallest oscillators of period 3 whose rotors have the following forms: .

▸ 71. [22] When looking for sequences of Life transition on a square grid, an asymmetrical solution will appear in eight different forms, because the grid has eight different symmetries. Furthermore, an asymmetrical periodic solution will appear in 8r different forms, if r is the length of the period.

Explain how to add further clauses so that essentially equivalent solutions will occur only once: Only “canonical forms” will satisfy the conditions.

72. [28] Oscillators of period 3 are particularly intriguing, because Life seems so inherently binary.

1. What are the smallest such oscillators (in terms of bounding box)?

2. Find period-3 oscillators of sizes 9 × n and 10 × n, with n odd, that have “fourfold symmetry”: The patterns are unchanged after left-right and/or up-down reflection. (Such patterns are not only pleasant to look at, they also are much easier to find, because we need only consider about one-fourth as many variables.)

3. What period-3 oscillators with fourfold symmetry have the most possible live cells, on grids of sizes 15 × 15, 15 × 16, and 16 × 16?

4. The period-3 oscillator shown here has another kind of four-way symmetry, because it’s unchanged after 90° rotation. (It was discovered in 1972 by Robert Wainwright, who called it “snake dance” because its stator involves four snakes.) What period-3 oscillators with 90° symmetry have the most possible live cells, on grids of sizes 15 × 15 and 16 × 16?

▸ 73. [21] (Mobile flipflops.) An oscillator of period 2 is called a flipflop, and the Life patterns of mobile flipflops are particularly appealing: Each cell is either blank (dead at every time t) or type A (alive when t is even) or type B (alive when t is odd). Every nonblank cell (i) has exactly three neighbors of the other type, and (ii) doesn’t have exactly two or three neighbors of the same type.

1. The blank cells of a mobile flipflop also satisfy a special condition. What is it?

2. Find a mobile flipflop on an 8 × 8 grid, with top row .

3. Find patterns that are mobile flipflops on m × n toruses for various m and n. (Thus, if replicated indefinitely, each one will tile the plane with an infinite mobile flipflop.) Hint: One solution has no blank cells whatsoever; another has blank cells like a checkerboard.

74. [M28] Continuing the previous exercise, prove that no nonblank cell of a finite mobile flipflop has more than one neighbor of its own type. (This fact greatly speeds up the search for finite mobile flipflops.) Can two type A cells be diagonally adjacent?

75. [M22] (Stephen Silver, 2000.) Show that a finite, mobile oscillator of period p ≥ 3 must have some cell that is alive more than once during the cycle.

76. [41] Construct a mobile Life oscillator of period 3.

77. [20] “Step X−1,” which precedes X_0 in (38), has the glider configuration instead of . What conditions on the still life X_5 will ensure that state X_0 is indeed reached? (We don’t want digestion to begin prematurely.)

78. [21] Find a solution to the four-step eater problem in (38) that works on a 7 × n grid, for some n, instead of 8 × 8.

79. [23] What happens if the glider meets the eater of (39) in its opposite phase (namely instead of )?

80. [21] To counteract the problem in the previous exercise, find an eater that is symmetrical when reflected about a diagonal, so that it eats both and . (You’ll have to go larger than 8 × 8, and you’ll have to wait longer for digestion.)

81. [21] Conway discovered a remarkable “spaceship,” where X_4 is X_0 shifted up 2:

Is there a left-right symmetrical still life that will eat such spaceships?

▸ 82. [22] (Light speed.) Imagine Life on an infinite plane, with all cells dead at time 0 except in the lower left quadrant. More precisely, suppose Xt = (xtij) is defined for all t ≥ 0 and all integers −∞ < i,j < +∞, and that x0ij = 0 whenever i > 0 or j > 0.

1. Prove that xtij = 0 whenever 0 ≤ t < max(i, j).

2. Furthermore xtij = 0 when 0 ≤ −i ≤ j and 0 ≤ t < i + 2j.

3. And xtij = 0 for 0 ≤ t < 2i + 2j, if i ≥ 0 and j ≥ 0. Hint: If xtij = 0 whenever i ≥ −j, prove that xtij = 0 whenever i > −j.

83. [21] According to the previous exercise, the earliest possible time that cell (i, j) can become alive, if all initial life is confined to the lower left quadrant of the plane, is at least
f(i,j) = [i≥0] + j[j≥0] + (i+j)[i+j≥0].

For example, when |i| ≤ 5 and |j| ≤ 5 the values of f(i, j) are shown at the right.

Let f*(i, j) be the actual minimum time at which cell (i, j) can be alive, for some such initial state. Devise a set of clauses by which a SAT solver can test whether or not f*(i_0, j_0) = f(i_0, j_0), given i_0 and j_0. (Such clauses make interesting benchmark tests.)

84. [33] Prove that f*(i, j) = f(i, j) in the following cases when j > 0: (a) i = j, i = j + 1, and i = j − 1. (b) i = 0 and i = −1. (c) i = 1 − j. (d) i = j − 2. (e) i = −2.

▸ 85. [39] A Garden of Eden is a state of Life that has no predecessor.

1. If the pattern of 92 cells illustrated here occurs anywhere within a bitmap X, verify that X is a Garden of Eden. (The gray cells can be either dead or alive.)

2. This “orphan” pattern, found with a SAT solver’s help, is the smallest that is currently known. Can you imagine how it was discovered?

86. [M23] How many Life predecessors does a random 10×10 bitmap have, on average?

87. [21] Explain why the clauses (42) represent Alice and Bob’s programs (40), and give a general recipe for converting such programs into equivalent sets of clauses.

88. [18] Satisfy (41) and (42) for 0 ≤ t < 6, and the 20 × 6 additional binary clauses that exclude multiple states, along with the “embarrassing” unit clauses (A3_6) ∧ (B3_6).

89. [21] Here’s a mutual-exclusion protocol once recommended in 1966. Does it work?

A0. Maybe go to A1.

A1. Set a ← 1, go to A2.

A2. If l go to A3, else to A5.

A3. If b go to A3, else to A4.

A4. Set l ← 0, go to A2.

A5. Critical, go to A6.

A6. Set a ← 0, go to A0.

B0. Maybe go to B1.

B1. Set b ← 1, go to B2.

B2. If l go to B5, else to B3.

B3. If a go to B3, else to B4.

B4. Set l ← 1, go to B2.

B5. Critical, go to B6.

B6. Set b ← 0, go to B0.

90. [20] Show that (43), (45), and (46) permit starvation, by satisfying (47) and (48).

91. [M21] Formally speaking, Alice is said to “starve” if there is (i) an infinite sequence of transitions X_0 → X_1 → … starting from the initial state X_0, and (ii) an infinite sequence @0, @1, ... of Boolean “bumps” that changes infinitely often, such that (iii) Alice is in a “maybe” or “critical” state only a finite number of times. Prove that this can happen if and only if there is a starvation cycle (47) as discussed in the text.

92. [20] Suggest O(r^2) clauses with which we can determine whether or not a mutual exclusion protocol permits a path X_0 → X_1 → … → Xr of distinct states.

93. [20] What clauses correspond to the term ¬Φ(X′) in (51)?

▸ 94. [21] Suppose we know that (X_0 → X_1 → … → Xr) ∧ ¬Φ(Xr) is unsatisfiable for 0 ≤ r ≤ k. What clauses will guarantee that Φ is invariant? (The case k = 1 is (51).)

95. [20] Using invariants like (50), prove that (45) and (46) provide mutual exclusion.

96. [22] Find all solutions to (52) when r = 2. Also illustrate the fact that invariants are extremely helpful, by finding a solution with distinct states X_0, X_1, ... , Xr and with r substantially greater than 2, if the clauses involving Φ are removed.

97. [20] Can states A6 and B6 occur simultaneously in Peterson’s protocol (49)?

▸ 98. [M23] This exercise is about proving the nonexistence of starvation cycles (47).

1. A cycle of states is called “pure” if one of the players is never bumped, and “simple” if no state is repeated. Prove that the shortest impure cycle, if any, is either simple or consists of two simple pure cycles that share a common state.

2. If Alice is starved by some cycle with protocol (49), we know that she is never in states A0 or A5 within the cycle. Show that she can’t be in A1, A2, or A6 either.

3. Construct clauses to test whether there exist states X_0 → X_1 → … → Xr, with X_0 arbitrary, such that (X_0X_1 ... Xk−1) is a starvation cycle for some k ≤ r.

4. Therefore we can conclude that (49) is starvation-free without much extra work.

99. [25] Th. Dekker devised the first correct mutual-exclusion protocol in 1965:

A0. Maybe go to A1.

A1. Set a ← 1, go to A2.

A2. If b go to A3, else to A6.

A3. If l go to A4, else to A2.

A4. Set a ← 0, go to A5.

A5. If l go to A5, else to A1.

A6. Critical, go to A7.

A7. Set l ← 1, go to A8.

A8. Set a ← 0, go to A0.

B0. Maybe go to B1.

B1. Set b ← 1, go to B2.

B2. If a go to B3, else to B6.

B3. If l go to B2, else to B4.

B4. Set b ← 0, go to B5.

B5. If l go to B1, else to B5.

B6. Critical, go to B7.

B7. Set l ← 0, go to B8.

B8. Set b ← 0, go to B0.

Use bounded model checking to verify its correctness.

100. [22] Show that the following protocol can starve one player but not the other:

A0. Maybe go to A1.

A1. Set a ← 1, go to A2.

A2. If b go to A2, else to A3.

A3. Critical, go to A4.

A4. Set a ← 0, go to A0.

B0. Maybe go to B1.

B1. Set b ← 1, go to B2.

B2. If a go to B3, else to B5.

B3. Set b ← 0, go to B4.

B4. If a go to B4, else to B1.

B5. Critical, go to B6.

B6. Set b ← 0, go to B0.

▸ 101. [31] Protocol (49) has the potential defect that Alice and Bob might both be trying to set the value of l at the same time. Design a mutual-exclusion protocol in which each of them controls two binary signals, visible to the other. Hint: The method of the previous exercise can be enclosed in another protocol.

102. [22] If Alice is setting a variable at the same time that Bob is trying to read it, we might want to consider a more stringent model under which he sees either 0 or 1, nondeterministically. (And if he looks k times before she moves to the next step, he might see 2k possible sequences of bits.) Explain how to handle this model of “flickering” variables by modifying the clauses of exercise 87.

103. [18] (Do this exercise by hand, it’s fun!) Find the 7×21 image whose tomographic sums are (r_1,...,r_7) = (1, 0, 13, 6, 12, 7, 19); (c_1,...,c_21) = (4, 3, 3, 4, 1, 6, 1, 3, 3, 3, 5, 1, 1, 5, 1, 5, 1, 5, 1, 1, 1); (a_1,...,a_27) = (0, 0, 1, 2, 2, 3, 2, 3, 3, 2, 3, 3, 4, 3, 2, 3, 3, 3, 4, 3, 2, 2, 1, 1, 1, 1, 1); (b_1,...,b_27) = (0, 0, 0, 0, 0, 1, 3, 3, 4, 3, 2, 2, 2, 3, 3, 4, 2, 3, 3, 3, 3, 3, 4, 3, 2, 1, 1).

104. [M21] For which m and n is it possible to satisfy the digital tomography problem with ad = bd = 1 for 0 < d < m + n? (Equivalently, when can m + n − 1 nonattacking bishops be placed on an m × n board?)

▸ 105. [M28] A matrix whose entries are {−1, 0, +1} is tomographically balanced if its row, column, and diagonal sums are all zero. Two binary images X = (xij) and X′=(xij′) clearly have the same row, column, and diagonal sums if and only if X − X′ is tomographically balanced.

1. Suppose Y is tomographically balanced and has m rows, n columns, and t occurrences of +1. How many m × n binary matrices X and X′ satisfy X − X′ = Y ?

2. Express the condition “Y is tomographically balanced” in terms of clauses, with the values {−1, 0, +1} represented respectively by the 2-bit codes {10, 00, 01}.

3. Count the number T(m, n) of tomographically balanced matrices, for m, n ≤ 8.

4. How many such matrices have exactly four occurrences of +1?

5. At most how many +1s can a 2n × 2n tomographically balanced matrix have?

6. True or false: The positions of the +1s determine the positions of the −1s.

106. [M20] Determine a generous upper bound on the possible number of different sets of input data {ri, cj, ad, bd} that might be given to a 25 × 30 digital tomography problem, by assuming that each of those sums independently has any of its possible values. How does this bound compare to 2^750?

▸ 107. [22] Basket weavers from the Tonga culture of Inhambane, Mozambique, have developed appealing periodic designs called “gipatsi patterns” such as this:

(Notice that an ordinary pixel grid has been rotated by 45°.) Formally speaking, a gipatsi pattern of period p and width n is a p × n binary matrix (xi,j) in which we have xi,1 = xi,n = 1 for 1 ≤ i ≤ p. Row i of the matrix is to be shifted right by i − 1 places in the actual pattern. The example above has p = 6, n = 13, and the first row of its matrix is 1111101111101. Such a pattern has row sums ri=Σj=1n xi,j for 1 ≤ i ≤ p and column sums cj=Σi=1p xi,j for 1 ≤ j ≤ n, as usual. By analogy with (53), it also has
ad=Σi+j≡d(modulo p)xi,j,1≤d≤p;bd=Σ2i+j≡d(modulo  2p)xi,j,1≤d≤2p.

1. What are the tomographic parameters ri, cj, ad, and bd in the example pattern?

2. Do any other gipatsi patterns have the same parameters?

108. [23] The column sums cj in the previous exercise are somewhat artificial, because they count black pixels in only a small part of an infinite line. If we rotate the grid at a different angle, however, we can obtain infinite periodic patterns for which each of Fig. 79’s four directions encounters only a finite number of pixels.

Design a pattern of period 6 in which parallel lines always have equal tomographic projections, by changing each of the gray pixels in the following diagram to either white or black:

▸ 109. [20] Explain how to find the lexicographically smallest solution x_1 ... xn to a satisfiability problem, using a SAT solver repeatedly. (See Fig. 80(a).)

110. [19] What are the lexicographically (first, last) solutions to waerden (3, 10; 96)?

111. [40] The lexicographically first and last solutions to the “Cheshire Tom” problem in Fig. 80 are based on the top-to-bottom-and-left-to-right ordering of pixels. Experiment with other pixel orderings—for example, try bottom-to-top-and-right-to-left.

112. [46] Exactly how many solutions does the tomography problem of Fig. 79 have?

▸ 113. [30] Prove that the digital tomography problem is NP-complete, even if the marginal sums r, c, a, b are binary: Show that an efficient algorithm to decide whether or not an n × n pixel image (xij) exists, having given 0–1 values of ri = ∑j xij, cj = ∑i xij, ad = ∑i+j=d+1 xij, and bd = ∑i−j=d−n xij, could be used to solve the binary tensor contingency problem of exercise 212(a).

114. [27] Each cell (i, j) of a given rectangular grid either contains a land mine (xi,j = 1) or is safe (xi,j = 0). In the game of Minesweeper, you are supposed to identify all of the hidden mines, by probing locations that you hope are safe: If you decide to probe a cell with xi,j = 1, the mine explodes and you die (at least virtually). But if xi,j = 0 you’re told the number ni,j of neighboring cells that contain mines, 0 ≤ ni,j ≤ 8, and you live to make another probe. By carefully considering these numeric clues, you can often continue with completely safe probes, eventually touching every mine-free cell.

For example, suppose the hidden mines happen to match the 25 × 30 pattern of the Cheshire cat (Fig. 79), and you start by probing the upper right corner. That cell turns out to be safe, and you learn that n1,30 = 0; hence it’s safe to probe all three neighbors of (1, 30). Continuing in this vein soon leads to illustration (α) below, which depicts information about cells (i, j) for 1 ≤ i ≤ 9 and 21 ≤ j ≤ 30; unprobed cells are shown in gray, otherwise the value of ni,j appears. From this data it’s easy to deduce that x1,24 = x2,24 = x3,25 = x4,25 = ··· = x9,26 = 1; you’ll never want to probe in those places, so you can mark such cells with X, arriving at state (β) since n3,24 = n5,25 = 4. Further progress downward to row 17, then leftward and up, leads without difficulty to state (γ). (Notice that this process is analogous to digital tomography, because you’re trying to reconstruct a binary array from information about partial sums.)

1. Now find safe probes for all thirteen of the cells that remain gray in (γ).

2. Exactly how much of the Cheshire cat can be revealed without making any unsafe guesses, if you’re told in advance that (i) x1,1 = 0? (ii) x1,30 = 0? (iii) x25,1 = 0? (iv) x25,30 = 0? (v) all four corners are safe? Hint: A SAT solver can help.

115. [25] Empirically estimate the probability that a 9×9 game of Minesweeper, with 10 randomly placed mines, can be won with entirely safe probes after the first guess.

116. [22] Find examples of Life flipflops for which X and X′ are tomographically equal.

117. [23] Given a sequence x = x_1 ... xn, let ν(2)x = x_1x_2 + x_2x_3 + ··· + xn−1xn. (A similar sum appears in the serial correlation coefficient, 3.3.2–(23).)

1. Show that, when x is a binary sequence, the number of runs of 1s in x can be expressed in terms of νx and ν(2)x.

2. Explain how to encode the condition ν(2)x ≤ r as a set of clauses, by modifying the cardinality constraints (20)–(21) of Bailleux and Boufkhad.

3. Similarly, encode the condition ν(2)x ≥ r.

118. [20] A tatami tiling is a covering by dominoes in which no three share a corner:

(Notice that is disallowed, but would be fine.) Explain how to use a SAT solver to find a tatami tiling that covers a given set of pixels, unless no such tiling exists.

119. [18] Let F = waerden (3, 3; 9) be the 32 clauses in (9). For which literal l is the reduced formula F | l smallest? Exhibit the resulting clauses.

120. [M20] True or false: F|L={C\L¯|C∈F and C ∩ L = Ø}, if L¯={l¯|l∈L}.

121. [21] Spell out the changes to the link fields in the data structures, by expanding the higher-level descriptions that appear in steps A3, A4, A7, and A8 of Algorithm A.

▸ 122. [21] Modify Algorithm A so that it finds all satisfying assignments of the clauses.

123. [17] Show the contents of the internal data structures L, START, and LINK when Algorithm B or Algorithm D begins to process the seven clauses R′ of (7).

▸ 124. [21] Spell out the low-level link field operations that are sketched in step B3.

▸ 125. [20] Modify Algorithm B so that it finds all satisfying assignments of the clauses.

126. [20] Extend the computation in (59) by one more step.

127. [17] What move codes m_1 ... md correspond to the computation sketched in (59), just before and after backtracking occurs?

128. [19] Describe the entire computation by which Algorithm D proves that Rivest’s clauses (6) are unsatisfiable, using a format like (59). (See Fig. 82.)

129. [20] In the context of Algorithm D, design a subroutine that, given a literal l, returns 1 or 0 according as l is or is not being watched in some clause whose other literals are entirely false.

130. [22] What low-level list processing operations are needed to “clear the watch list for x¯k” in step D6?

▸ 131. [30] After Algorithm D exits step D3 without finding any unit clauses, it has examined the watch lists of every free variable. Therefore it could have computed the lengths of those watch lists, with little additional cost; and information about those lengths could be used to make a more informed decision about the variable that’s chosen for branching in step D4. Experiment with different branching heuristics of this kind.

▸ 132. [22] Theorem 7.1.1K tells us that every 2SAT problem can be solved in linear time. Is there a sequence of 2SAT clauses for which Algorithm D takes exponential time?

▸ 133. [25] The size of a backtrack tree such as Fig. 82 can depend greatly on the choice of branching variable that is made at every node.

1. Find a backtrack tree for waerden (3, 3; 9) that has the fewest possible nodes.

2. What’s the largest backtrack tree for that problem?

134. [22] The BIMP tables used by Algorithm L are sequential lists of dynamically varying size. One attractive way to implement them is to begin with every list having capacity 4 (say); then when a list needs to become larger, its capacity can be doubled.

Adapt the buddy system (Algorithm 2.5R) to this situation. (Lists that shrink when backtracking needn’t free their memory, since they’re likely to grow again later.)

▸ 135. [16] The literals l′ in BIMP(l) are those for which l → l′ in the “implication digraph” of a given satisfiability problem. How can we easily find all of the literals l″ such that l″ → l, given l?

136. [15] What pairs will be in TIMP(3¯), before and after x_5 is set to zero with respect to the clauses (9) of waerden (3, 3; 9), assuming that we are on decision level d = 0?

137. [24] Spell out in detail the processes of (a) removing a variable X from the free list and from all pairs in TIMP lists (step L7 of Algorithm L), and of (b) restoring it again later (step L12). Exactly how do the data structures change?

▸ 138. [20] Discuss what happens in step L9 of Algorithm L if we happen to have both v¯∈BIMP(u¯) and u¯∈BIMP(v¯).

139. [25] (Compensation resolvents.) If w ∈ BIMP(v), the binary clause u ∨ v implies the binary clause u ∨ w, because we can resolve u ∨ v with v¯∨w. Thus step L9 could exploit each new binary clause further, by appending w as well as v to BIMP(u¯), for all such w. Discuss how to do this efficiently.

140. [21] The FORCE, BRANCH, BACKF, and BACKI arrays in Algorithm L will obviously never contain more than n items each. Is there a fairly small upper bound on the maximum possible size of ISTACK?

141. [18] Algorithm L might increase ISTAMP so often that it overflows the size of the IST(l) fields. How can the mechanism of (63) avoid bugs in such a case?

142. [24] Algorithms A, B, and D can display their current progress by exhibiting a sequence of move codes m_1 ... md such as (58) and (60); but Algorithm L has no such codes. Show that an analogous sequence m_1 ... mF could be printed in step L2, if desired. Use the codes of Algorithm D; but extend them to show mj = 6 (or 7) if Rj−1 is a true (or false) literal whose value was found to be forced by Algorithm X, or forced by being a unit clause in the input.

▸ 143. [30] Modify Algorithm L so that it will apply to nonempty clauses of any size. Call a clause big if its size is greater than 2. Instead of TIMP tables, represent every big clause by ‘KINX’ and ‘CINX’ tables: Every literal l has a sequential list KINX(l) of big clause numbers; every big clause c has a sequential list CINX(c) of literals; c is in KINX(l) if and only if l is in CINX(c). The current number of active clauses containing l is indicated by KSIZE(l); the current number of active literals in c is indicated by CSIZE(c).

144. [15] True or false: If l doesn’t appear in any clause, h′(l) = 0.1 in (65).

145. [23] Starting with h(l) = 1 for each of the 18 literals l in waerden (3, 3; 9), find successively refined estimates h′(l), h″(l), ... , using (65) with respect to the 32 ternary clauses (9). Then, assuming that x_5 has been set false as in exercise 136, and that the resulting binary clauses 13, 19, 28, 34, 37, 46, 67, 79 have been included in the BIMP tables, do the same for the 16 literals that remain at depth d = 1.

146. [25] Suggest an alternative to (64) and (65) for use when Algorithm L has been extended to nonternary clauses as in exercise 143. (Strive for simplicity.)

147. [05] Evaluate Cmax in (66) for d = 0, 1, 10, 20, 30, using the default C_0 and C_1.

148. [21] Equation (66) bounds the maximum number of candidates using a formula that depends on the current depth d, but not on the total number of free variables. The same cutoffs are used in problems with any number of variables. Why is that a reasonable strategy?

▸ 149. [26] Devise a data structure that makes it convenient to tell whether a given variable x is a “participant” in Algorithm L.

150. [24] Continue the text’s story of lookahead in waerden (3, 3; 9): What happens at depth d = 1 when l ← 7 and T ← 22 (see (70)), after literal 4 has become proto true? (Assume that no double-lookahead is done.)

▸ 151. [26] The dependency digraph (68) has 16 arcs, only 8 of which are captured in the subforest (69). Show that, instead of (70), we could actually list the literals l and give them offsets o(l) in such a way that u appears before v in the list and has o(u) > o(v) if and only if v → u in (68). Thus we could capture all 16 dependencies via levels of truth.

152. [22] Give an instance of 3SAT for which no free “participants” are found in step X3, yet all clauses are satisfied. Also describe an efficient way to verify satisfaction.

153. [17] What’s a good way to weed out unwanted candidates in step X3, if C>Cmax?

154. [20] Suppose we’re looking ahead with just four candidate variables, {a, b, c, d}, and that they’re related by three binary clauses (a∨b¯)∧(a∨c¯)∧(c∨d¯). Find a subforest and a sequence of truth levels to facilitate lookaheads, analogous to (69) and (70).

155. [32] Sketch an efficient way to construct the lookahead forest in step X4.

156. [05] Why is a pure literal a special case of an autarky?

157. [10] Give an example of an autarky that is not a pure literal.

158. [15] If l is a pure literal, will Algorithm X discover it?

159. [M17] True or false: (a) A is an autarky for F if and only if F | A ⊆ F. (b) If A is an autarky for F and A′ ⊆ A, then A \ A′ is an autarky for F | A′.

160. [18] (Black and white principle.) Consider any rule by which literals have been colored white, black, or gray in such a way that l is white if and only if l¯ is black. (For example, we might say that l is white if it appears in fewer clauses than l¯.)

1. Suppose every clause of F that contains a white literal also contains a black literal. Prove that F is satisfiable if and only if its all-gray clauses are satisfiable.

2. Explain why this metaphor is another way to describe the notion of an autarky.

▸ 161. [21] (Black and blue principle.) Now consider coloring literals either white, black, orange, blue, or gray, in such a way that l is white if and only if l¯ is black, and l is orange if and only if l¯ is blue. (Hence l is gray if and only if l¯ is gray.) Suppose further that F is a set of clauses in which every clause containing a white literal also contains either a black literal or a blue literal (or both). Let A = {a_1,...,ap} be the black literals and let L = {l_1,...,lq} be the blue literals. Also let F′ be the set of clauses obtained by adding p additional clauses (l¯1∨⋅⋅⋅∨l¯q∨aj) to F, for 1 ≤ j ≤ p.

1. Prove that F is satisfiable if and only if F′ is satisfiable.

2. Restate and simplify that result in the case that p = 1.

3. Restate and simplify that result in the case that q = 1.

4. Restate and simplify that result in the case that p = q = 1. (In this special case, (l¯∨a) is called a blocked binary clause.)

162. [21] Devise an efficient way to discover all of the (a) blocked binary clauses (l¯∨a) and (b) size-two autarkies A = {a, a′} of a given kSAT problem F.

▸ 163. [M25] Prove that the following recursive procedure R(F) will solve any n-variable 3SAT problem F with at most O(ϕn) executions of steps R1, R2, or R3:

R1. [Check easy cases.] If F = Ø, return true. If Ø ∈ F, return false. Otherwise let {l_1,...,ls} ∈ F be a clause of minimum size s.

R2. [Check autarkies.] If s = 1 or if {ls} is an autarky, set F ← F | ls and return to R1. Otherwise if {l¯s,ls−1} is an autarky, set F←F|l¯s,ls−1 and return to R1.

R3. [Recurse.] If R(F | ls) is true, return true. Otherwise set F←F|l¯s, s ← s − 1, and go back to R2.

164. [M30] Continuing exercise 163, bound the running time when F is kSAT.

▸ 165. [26] Design an algorithm to find the largest positive autarky A for a given F, namely an autarky that contains only positive literals. Hint: Warm up by finding the largest positive autarky for the clauses (123¯, 125¯, 1¯3¯4¯, 136¯, 14¯5, 156, 2¯35, 24¯6, 345, 3¯56}.

166. [30] Justify the operations of step X9. Hint: Prove that an autarky can be constructed, if w = 0 after (72) has been performed.

▸ 167. [21] Justify step X11 and the similar use of X12 in step X6.

168. [26] Suggest a way to choose the branch literal l in step L3, based on the heuristic scores H(l) that were compiled by Algorithm X in step L2. Hint: Experience shows that it’s good to have both H(l) and H(l¯) large.

▸ 169. [HM30] (T. Ahmed, O. Kullmann.) Excellent results have been obtained in some problems when the branch variable in step L3 is chosen to minimize the quantity τ(H(l), H(l¯)), where τ(a, b) is the positive solution to τ−a + τ−b = 1. (For example, τ(1, 2) = ϕ ≈ 1.62 and τ(2,2)=21/2≈1.63, so we prefer (1, 2) to (2,2).) Given a list of pairs of positive numbers (a_1, b_1), ... , (as, bs), what’s an efficient way to determine an index j that minimizes τ(aj, bj), without computing logarithms?

170. [25] (Marijn Heule, 2013.) Show that Algorithm L solves 2SAT in linear time.

171. [20] What is the purpose of DFAIL in Algorithm Y?

172. [21] Explain why ‘+LO[j]’ appears in step Y2’s formula for DT.

173. [40] Use an implementation of Algorithm L to experiment with random 3SAT problems such as rand (3, 2062, 500, 314). Examine the effects of such things as (i) disabling double lookahead; (ii) disabling “wraparound,” by changing the cases j = S and j^=S in X7 and Y4 so that they simply go to X6 and Y3; (iii) disabling the lookahead forest, by letting all candidate literals have null PARENT; (iv) disabling compensation resolvents in step L9; (v) disabling “windfalls” in (72); (vi) branching on a random free candidate l in L3, instead of using the H scores as in exercise 168; or (vii) disabling all lookahead entirely as in “Algorithm L^0.”

174. [15] What’s an easy way to accomplish (i) in the previous exercise?

175. [32] When Algorithm L is extended to nonternary clauses as in exercise 143, how should Algorithms X and Y also change? (Instead of using (64) and (65) to compute a heuristic for preselection, use the much simpler formula in answer 146. And instead of using h(u)h(v) in (67) to estimate the weight of a ternary clause that will be reduced to binary, consider a simulated reduced clause of size s ≥ 2 to have weight Ks ≈ γs−2, where γ is a constant (typically 0.2).)

176. [M25] The “flower snark” Jq is a cubic graph with 4q vertices tj, uj, vj, wj, and 6q edges tj — tj+1, tj — uj, uj — vj, uj — wj, vj — wj+1, wj — vj+1, for 1 ≤ j ≤ q, with subscripts treated modulo q. Here, for example, are J_5 and its line graph L(J_5):

1. Give labels aj, bj, cj, dj, ej, and fj to the edges of Jq, for 1 ≤ j ≤ q. (Thus aj denotes tj — tj+1 and bj denotes tj — uj, etc.) What are the edges of L(Jq)?

2. Show that χ(Jq) = 2 and χ(L(Jq)) = 3 when q is even.

3. Show that χ(Jq) = 3 and χ(L(Jq)) = 4 when q is odd. Note: Let fsnark(q) denote the clauses (15) and (16) that correspond to 3-coloring L(Jq), together with (b1,1) ∧ (c1,2) ∧ (d1,3) to set the colors of (b_1,c_1,d_1) to (1, 2, 3). These clauses make excellent benchmark tests for SAT solvers.

177. [HM26] Let Iq be the number of independent sets of the flower snark line graph L(Jq). Compute Iq for 1 ≤ q ≤ 8, and determine the asymptotic growth rate.

▸ 178. [M23] When Algorithm B is presented with the unsatisfiable clauses fsnark(q) of exercise 176, with q odd, its speed depends critically on the ordering of the variables.

Show that the running time is Θ(2q) when the variables are considered in the order
a1,1 a1,2 a1,3 b1,1 b1,2 b1,3 c1,1 c1,2 c1,3 d1,1 d1,2 d1,3 e1,1 e1,2 e1,3 f1,1 f1,2 f1,3 a2,1 a2,2 a2,3 . . . ;
but much, much more time is needed when the order is
a1,1 b1,1 c1,1 d1,1 e1,1 f1,1 a2,1 b2,1 c2,1 d2,1 e2,1 f2,1 . . . aq,1 bq,1 cq,1 dq,1 eq,1 fq,1 a1,2 b1,2 c1,2 . . . .

179. [25] Show that there are exactly 4380 ways to fill the 32 cells of the 5-cube with eight 4-element subcubes. For example, one such way is to use the subcubes 000**, 001**, ... , 111**, in the notation of 7.1.1 (29); a more interesting way is to use

0*0*0,1*0*0,**001,**110,*010*,*110*0**11,1**11.

What does this fact tell you about the value of q_8 in Fig. 83?

▸ 180. [25] Explain how to use BDDs to compute the numbers Qm that underlie Fig. 83. What is max0≤m≤80 Qm?

▸ 181. [25] Extend the idea of the previous exercise so that it is possible to determine the probability distributions Tm of Fig. 84.

182. [M16] For which values of m in Fig. 84 does Tm have a constant value?

183. [M30] Discuss the relation between Figs. 85 and 86.

184. [M20] Why does (77) characterize the relation between q^m and qm?

185. [M20] Use (77) to prove the intuitively obvious fact that q^m≥qm.

186. [M21] Use (77) to reduce Σm q^m and Σm (2m+1)q^m to (78) and (79).

187. [M20] Analyze random satisfiability in the case k = n: What are Sn,n and S^n,n?

▸ 188. [HM25] Analyze random 1SAT, the case k = 1: What are S1,n and S^1,n?

189. [27] Apply BDD methods to random 3SAT problems on 50 variables. What is the approximate BDD size after m distinct clauses have been ANDed together, as m grows?

190. [M20] Exhibit a Boolean function of 4 variables that can’t be expressed in 3CNF. (No auxiliary variables are allowed: Only x_1, x_2, x_3, and x_4 may appear.)

191. [M25] How many Boolean functions of 4 variables can be expressed in 3CNF?

▸ 192. [HM21] Another way to model satisfiability when there are N equally likely clauses is to study S(p), the probability of satisfiability when each clause is independently present with probability p.

1. Express S(p) in terms of the numbers Qm=(Nm)qm.

2. Assign uniform random numbers in [0 .. 1) to each clause; then at time t, for 0 ≤ t ≤ N, consider all clauses that have been assigned a number less than t/N. (Approximately t clauses will therefore be selected, when N is large.) Show that S¯k,n=∫0N Sk,n(t/N)dt, the expected amount of time during which the chosen clauses remain satisfiable, is very similar to the satisfiability threshold Sk,n of (76).

193. [HM48] Determine the satisfiability threshold (81) of random 3SAT. Is it true that lim infn→∞ S3,n/n = lim supn→∞ S3,n/n? If so, is the limit ≈ 4.2667?

194. [HM49] If α < lim infn→∞ S3,n/n, is there a polynomial-time algorithm that is able to satisfy ⌊αn⌋ random 3SAT clauses with probability ≥ δ, for some δ > 0?

195. [HM21] (J. Franco and M. Paull, 1983.) Use the first moment principle MPR–(21) to prove that ⌊(2k ln 2)n⌋ random kSAT clauses are almost always unsatisfiable. Hint: Let X = ∑x [x satisfies all clauses], summed over all 2n binary vectors x = x_1 ... xn.

▸ 196. [HM25] (D. B. Wilson.) A clause of a satisfiability problem is “easy” if it contains one or more variables that don’t appear in any other clauses. Prove that, with probability 1 − O(n−2∈), a kSAT problem that has m = ⌊αn⌋ random clauses contains (1 − (1 − e−kα)k)m + O(n1/2+∈) easy ones. (For example, about 0.000035n of the 4.27n clauses in a random 3SAT problem near the threshold will be easy.)

197. [HM21] Prove that the quotient q(a,b,A,B,n)=((a+b)nan)((A+B)nAn)/((a+b+A+B)n(a+A)n) is O(n−1/2) as n → ∞, if a, b, A, B > 0.

▸ 198. [HM30] Use exercises 196 and 197 to show that the phase transition in Fig. 89 is not extremely abrupt: If S3(m,n)>23 and S3(m′,n)<13, prove that m′=m+Ω(n).

199. [M21] Let p(t, m, N) be the probability that t specified letters each occur at least once within a random m-letter word on an N-letter alphabet.

1. Prove that p(t,m,N)≤mt¯/Nt.

2. Derive the exact formula p(t,m,N)=Σk (tk)(−1)k(N−k)m/Nm.

3. And p(t,m,N)/t!=(tt)(mt)/Nt−{t+1t}(mt+1)/Nt+1+{t+2t}(mt+2)/Nt+2−….

▸ 200. [M21] Complete the text’s proof of (84) when c < 1:

1. Show that every unsatisfiable 2SAT formula contains clauses of a snare.

2. Conversely, are the clauses of a snare always unsatisfiable?

3. Verify the inequality (89). Hint: See exercise 199.

201. [HM29] The t-snake clauses specified by a chain (l_1,...,l2t−1) can be written (l¯i∨li+1) for 0 ≤ i < 2t, where l0=l¯t and subscripts are treated mod 2t.

1. Describe all ways to set two of the l’s so that (x¯1∨x2) is one of those 2t clauses.

2. Similarly, set three l’s in order to obtain (x¯1∨x2) and (x¯2∨x3).

3. Also set three to obtain both (x¯0∨x1) and (x¯t−1∨xt); here x¯0≡xt and t > 2.

4. How can the clauses (x¯i∨xi+1) for 0 ≤ i < t all be obtained by setting t of the l’s?

5. In general, let N(q, r) be the number of ways to choose r of the standard clauses (x¯i∨xi+1), which involve exactly q of the variables {x_1,...,x2t−1}, and to set q values of {l_1,...,l2t−1} in order to obtain the r chosen clauses. Evaluate N(2, 1).

6. Similarly, evaluate N(3, 2), N(t, t), and N(2t − 1, 2t).

7. Show that the probability pr in (95) is ≤Σq N(q,r)/(2qnq¯).

8. Therefore the upper bound (96) is valid.

202. [HM21] This exercise amplifies the text’s proof of Theorem C when c > 1.

1. Explain the right-hand side of Eq. (93).

2. Why does (97) follow from (95), (96), and the stated choices of t and m?

▸ 203. [HM33] (K. Xu and W. Li, 2000.) Beginning with the n graph-coloring clauses (15), and optionally the n(d2) exclusion clauses (17), consider using randomly generated binary clauses instead of (16). There are mq random binary clauses, obtained as m independent sets of q clauses each, where every such set is selected by choosing distinct vertices u and v, then choosing q distinct binary clauses (u¯i∨v¯j) for 1 ≤ i, j ≤ d. (The number of different possible sequences of random clauses is therefore exactly ((n2)(d2q))m and each sequence is equally likely.) This method of clause generation is known as “Model RB”; it generalizes random 2SAT, which is the case d = 2 and q = 1.

Suppose d = nα and q = pd^2, where we require 12<α<1 and 0≤p≤12. Also let m = rn ln d. For this range of the parameters, we will prove that there is a sharp threshold of satisfiability: The clauses are unsatisfiable q.s., as n → ∞, if r ln(1 − p) + 1 < 0; but they are satisfiable a.s. if r ln(1 − p) + 1 > 0.

Let X(j_1,...,jn) = [all clauses are satisfied when each ith variable v has vji=1]; here 1 ≤ j_1,...,jn ≤ d. Also let X=Σ1≤j1,...,jn≤dX(j1,...,jn). Then X = 0 if and only if the clauses are unsatisfiable.

1. Use the first moment principle to prove that X = 0 q.s. when r ln(1 − p)+1 < 0.

2. Find a formula for ps = Pr(X(j_1,...,jn) = 1 | X(1,..., 1) = 1), given that exactly s of the colors {j_1,...,jn} are equal to 1.

3. Use (b) and the conditional expectation inequality MPR–(24) to prove that X > 0 a.s. if

Σs=0n(ns)(1d)s(1−1d)n−s(1+p1−ps2n2)m→1as n→∞.

4. Letting ts denote the term for s in that sum, prove that Σs=03n/dts≈1.

5. Suppose r ln(1 − p) + 1 = ∈ > 0, where ∈ is small. Show that the terms ts first increase, then decrease, then increase, then decrease again, as s grows from 0 to n. Hint: Consider the ratio xs = ts+1/ts.

6. Finally, prove that ts is exponentially small for 3n/d ≤ s ≤ n.

▸ 204. [28 ] Figure 89 might suggest that 3SAT problems on n variables are always easy when there are fewer than 2n clauses. We shall prove, however, that any set of m ternary clauses on n variables can be transformed mechanically into another set of ternary clauses on N = O(m) variables in which no variable occurs more than four times. The transformed problem has the same number of solutions as the original one; thus it isn’t any simpler, although (with at most 4N literals) it has at most 43Nclauses.

1. First replace the original m clauses by m new clauses (X_1∨X_2∨X_3), ... ,(X3m−2∨X3m−1 ∨ X3m), on 3m new variables, and show how to add 3m clauses of size 2 so that the resulting 4m clauses have exactly as many solutions as the original.

2. Construct ternary clauses that have a unique solution, yet no variable occurs more than four times.

3. Use (a) and (b) to prove the N-variable result claimed above.

205. [26 ] If F and F′ are sets of clauses, let F ⊔ F′ stand for any other set obtained from F ∪ F′ by replacing one or more clauses C of F by x∨C and one or more clauses C′ of F′ by x¯∨C′, where x is a new variable. Then F ⊔ F′ is unsatisfiable whenever F and F′ are both unsatisfiable. For example, if F = {∈}> and F′={1,1¯}, then F ⊔ F′ is either {2,12¯,1¯2¯} or {2,1,1¯2¯} or {2,12¯,1¯}.

1. Construct 16 unsatisfiable ternary clauses on 15 variables, where each variable occurs at most four times.

2. Construct an unsatisfiable 4SAT problem in which every variable occurs at most five times.

206. [M22] A set of clauses is minimally unsatisfiable if it is unsatisfiable, yet becomes satisfiable if any clause is deleted. Show that, if F and F′ have no variables in common, then F⊔F′ is minimally unsatisfiable if and only if F and F′ are minimally unsatisfiable.

207. [25] Each of the literals {1,1¯,2,2¯,3,3¯,4,4¯} occurs exactly thrice in the eight unsatisfiable clauses (6). Construct an unsatisfiable 3SAT problem with 15 variables in which each of the 30 literals occurs exactly twice. Hint: Consider {1¯2,2¯3,3¯1,123,1¯2¯3¯}.

208. [25] Via exercises 204(a) and 207, show that any 3SAT problem can be transformed into an equivalent set of ternary clauses where every literal occurs just twice.

209. [25] (C. A. Tovey.) Prove that every k SAT formula in which no variable occurs more than k times is satisfiable. (Thus the limits on occurrences in exercises 204–208 cannot be lowered, when k = 3 and k = 4.) Hint: Use the theory of bipartite matching.

210. [M36] But the result in the previous exercise can be improved when k is large. Use the Local Lemma to show that every 7SAT problem with at most 13 occurrences of each variable is satisfiable.

211. [30] (R. W. Irving and M. Jerrum, 1994.) Use exercise 208 to reduce 3SAT to the problem of list coloring a grid graph of the form KN □ K_3. (Hence the latter problem, which is also called latin rectangle construction, is NP-complete.)

212. [32] Continuing the previous exercise, we shall reduce grid list coloring to another interesting problem called partial latin square construction. Given three n × n binary matrices (rik), (cjk), (pij), the task is to construct an n × n array (Xij) such that Xij is blank when pij = 0, otherwise Xij = k for some k with rik = cjk = 1; furthermore the nonblank entries must be distinct in each row and column.

1. Show that this problem is symmetrical in all three coordinates: It’s equivalent to constructing a binary n × n × n tensor (xijk) such that x∗jk = cjk, xi∗k = rik, and xij∗ = pij, for 1 ≤ i, j, k ≤ n, where ‘∗’ denotes summing an index from 1 to n. (Therefore it is also known as the binary n × n × n contingency problem, given n^2 row sums, n^2 column sums, and n^2 pile sums.)

2. A necessary condition for solution is that c∗k = r∗k, cj∗ = p∗j, and ri∗ = pi∗. Exhibit a small example where this condition is not sufficient.

3. If M < N, reduce KM □ KN list coloring to the problem of KN □ KN list coloring.

4. Finally, explain how to reduce KN □ KN list coloring to the problem of constructing an n × n partial latin square, where n = N + ΣI,J|L(I, J)|. Hint: Instead of considering integers 1 ≤ i, j, k ≤ n, let i, j, k range over a set of n elements. Define pij = 0 for most values of i and j; also make rik = cik for all i and k.

▸ 213. [M20] Experience with the analyses of sorting algorithms in Chapter 5 suggests that random satisfiability problems might be modeled nicely if we assume that, in each of m independent clauses, the literals xj and x¯j occur with respective probabilities p and q, independently for 1 ≤ j ≤ n, where p + q ≤ 1. Why is this not an interesting model as n → ∞, when p and q are constant? Hint: What is the probability that x_1 ... xn = b_1 ... bn satisfies all of the clauses, when b_1 ... bn is a given binary vector?

214. [HM38] Although the random model in the preceding exercise doesn’t teach us how to solve SAT problems, it does lead to interesting mathematics: Let 0 < p < 1 and consider the recurrence

T0=0;Tn=n+2Σk=0n−1(nk)pk(1−9)n−kTk,for n>0.

1. Find a functional relation satisfied by T(z)=Σn=0∞Tnzn/n!.

2. Deduce that we have T(z)=zezΣm=0∞(2p)mΠk=0m−1(1−e−pk(1−p)z)

3. Hence, if p ≠ 1/2, we can use Mellin transforms (as in the derivation of 5.2.2–(50)) to show that Tn = Cpnα(1 + δ(n) + O(1/n)) + n/(1 − 2p), where α = 1/lg(1/p), Cp is a constant, and δ is a small “wobble” with δ(n) = δ(pn).

▸ 215. [HM28] What is the expected profile of the search tree when a simple backtrack procedure is used to find all solutions to a random 3SAT problem with m independent clauses on n variables? (There is a node on level l for every partial solution x_1 ... xl that doesn’t contradict any of the clauses.) Compute these values when m = 200 and n = 50. Also estimate the total tree size when m = αn, for fixed α as n → ;∞.

216. [HM38] (P. W. Purdom, Jr., and C. A. Brown.) Extend the previous exercise to a more sophisticated kind of backtracking, where all choices forced by unit clauses are pursued before two-way branching is done. (The “pure literal rule” is not exploited, however, because it doesn’t find all solutions.) Prove that the expected tree size is greatly reduced when m = 200 and n = 50. (An upper bound is sufficient.)

217. [20] True or false: If A and B are arbitrary clauses that are simultaneously satisfiable, and if l is any literal, then the clause C=(A∪B)∖{l,l¯} is also satisfiable. (We’re thinking here of A, B, and C as sets of literals, not as disjunctions of literals.)

218. [20] Express the formula (x∨A)∧(x¯∨B) in terms of the ternary operator u? v: w.

▸ 219. [M20] Formulate a general definition of the resolution operator C = C′ ◊ C″ that (i) agrees with the text’s definition when C′ = x ∨ A′ and C"=x¯∨A"; (ii) applies to arbitrary clauses C′ and C″; (iii) has the property that C′ ∧ C″ implies C′ ◊ C″.

220. [M24] We say that clause C subsumes clause C′, written C ⊆ C′, if C = ℘ or if C′ ≠ ℘ and every literal of C appears in C′.

1. True or false: C ⊆ C′ and C′ ⊆ C″ implies C ⊆ C″.

2. True or false: (C ∨ α) ◊ (C′ ∨ α′) ⊆ (C ◊ C′) ∨ α ∨ α′, with ◊ as in exercise 219.

3. True or false: C′ ⊆ C″ implies C ◊ C′ ⊆ C ◊ C″.

4. The notation C_1,..., Cm ⊢ C means that a resolution chain C_1,..., Cm+r exists with Cm+r ⊆ C, for some r ≥ 0. Show that we might have C_1,..., Cm ⊢ C even though C cannot be obtained from {C_1,..., Cm} by successive resolutions (104).

5. Prove that if C1⊆C1′,⋅⋅⋅,Cm⊆Cm′, and C1′,...,Cm′⊢C, then C_1,..., Cm ⊢ C.

6. Furthermore C_1,..., Cm⊢ C implies C_1 ∨ α_1,..., Cm ∨ αm ⊢ C ∨ α_1 ∨ ··· ∨ αm.

221. [16] Draw the search tree analogous to Fig. 81 that is implicitly traversed when Algorithm A is applied to the unsatisfiable clauses {12,2,2¯}. Explain why it does not correspond to a resolution refutation that is analogous to Fig. 91.

222. [M30] (Oliver Kullmann, 2000.) Prove that, for every clause C in a satisfiability problem F , there is an autarky satisfying C if and only if C cannot be used as the label of a source vertex in any resolution refutation of F .

223. [HM40] Step X9 deduces a binary clause that cannot be derived by resolution (see exercise 166). Prove that, nevertheless, the running time of Algorithm L on unsatisfiable input will never be less than the length of a shortest treelike refutation.

224. [M20] Given a resolution tree that refutes the axioms F|x¯, show how to construct a resolution tree of the same size that either refutes the axioms F or derives the clause {x} from F without resolving on the variable x.

▸ 225. [M31] (G. S. Tseytin, 1966.) If T is any resolution tree that refutes a set of axioms F , show how to convert it to a regular resolution tree Tr that refutes F , where Tr is no larger than T .

226. [M20] If α is a node in a refutation tree, let C(α) be its label, and let ∥α∥ denote the number of leaves in its subtree. Show that, given a refutation tree with N leaves, the Prover can find a node with ∥α∥ ≤ N/2s for which the current assignment falsifies C(α), whenever the Delayer has scored s points in the Prover–Delayer game.

227. [M27] Given an extended binary tree, exercise 7.2.1.6–124 explains how to label each node with its Horton–Strahler number. For example, the nodes at depth 2 in Fig. 91 are labeled 1, because their children have the labels 1 and 0; the root is labeled 3.

Prove that the maximum score that the Delayer can guarantee, when playing the Prover–Delayer game for a set of unsatisfiable clauses F , is equal to the minimum possible Horton–Strahler root label in a tree refutation of F .

▸ 228. [M21] Stålmarck’s refutation of (99)–(101) actually obtains ∊ without using all of the axioms! Show that only about 1/3 of those clauses are sufficient for unsatisfiability.

▸ 229. [M21] Continuing exercise 228, prove also that the set of clauses (99), (100′), (101) is unsatisfiable, where (100′) denotes (100) restricted to the cases i ≤ k and j < k.

230. [M22] Show that the clauses with i ≠ j in the previous exercise form a minimal unsatisfiable set: Removing any one of them leaves a satisfiable remainder.

231. [M30] (Sam Buss.) Refute the clauses of exercise 229 with a resolution chain of length O(m^3). Hint: Derive the clauses Gij = (xij∨xi(j+1)∨···∨xim) for 1 ≤ i ≤ j ≤ m.

▸ 232. [M28] Prove that the clauses fsnark(q) of exercise 176 can be refuted by treelike resolution in O(q^6) steps.

233. [16] Explain why (105) satisfies (104), by exhibiting j(i) and k(i) for 9 ≤ i ≤ 22.

234. [20] Show that the Delayer can score at least m points against any Prover who tries to refute the pigeonhole clauses (106) and (107).

▸ 235. [30] Refute those pigeonhole clauses with a chain of length m(m+3)2m−2.

236. [48] Is the chain in the previous exercise as short as possible?

▸ 237. [28] Show that a polynomial number of steps suffice to refute the pigeonhole clauses (106), (107), if the extended resolution trick is used to append new clauses.

238. [HM21] Complete the proof of Lemma B. Hint: Make r ≤ ρ−b when W = b.

▸ 239. [M21] What clauses α_0 on n variables make ∥α_0 ⊢ ∊∥ as large as possible?

240. [HM23] Choose integers fij ∊ {1,...,m} uniformly at random, for 1 ≤ i ≤ 5 and 0 ≤ j ≤ m, and let G_0 be the bipartite graph with edges aj −−bk if and only if k ∊ {f_1j,..., f_5j}. Show that Pr(G_0satisfies the strong expansion condition (108)) ≥ 1/2.

241. [20] Prove that any set of at most m/3000 pigeons can be matched to distinct holes, under the restricted pigeonhole constraints G_0 of Theorem B.

242. [M20] The pigeonhole axioms (106) and (107) are equivalent to the clauses (15) and (16) that arise if we try to color the complete graph Km+1 with m colors.

Suppose we include further axioms corresponding to (17), namely

(x¯jk∨x¯jk′).for 0≤j≤m⁢and1≤k<k′≤m.

Does Theorem B still hold, or do these additional axioms decrease the refutation width?

243. [HM31] (E. Ben-Sasson and A. Wigderson.) Let F be a set of ⌊αn⌋ random 3SAT clauses on n variables, where α > 1/e is a given constant. For any clause C on those variables, define μ(C) = min{ |F′| | F′ ⊆ F and F′ ⊢ C}. Also let V (F′) denote the variables that occur in a given family of clauses F′.

1. Prove that |V (F′)| ≥ |F′| a.s., when F′ ⊆F and |F′| ≤ n/(2αe^2).

2. Therefore either F is satisfiable or μ(∊) > n/(2αe^2), a.s.

3. Let n′ = n/(1000000α^4), and assume that n′ ≥ 2. Prove that 2|V(F′)|−3|F′| ≥ n′/4 q.s., when F′ ⊆ F and n′/2 ≤ |F′|< n′.

4. Consequently eitherF is satisfiable or w(F ⊢ ∊) ≥ n′/4, a.s.

244. [M20] If A is a set of variables, let [A]0 or [A]1 stand for the set of all clauses that can be formed from A with an even or odd number of negative literals, respectively; each clause should involve all of the variables. (For example, [{1, 2, 3}]1 = {123¯,12¯3,1¯23,1¯2¯3¯}.) If A and B are disjoint, express [A ∪B]0 in terms of the sets [A]0, [A]1, [B]0, [B]1.

▸ 245. [M27] Let G be a connected graph whose vertices v ∊ V have each been labeled 0 or 1, where the sum of all labels is odd. We will construct clauses on the set of variables euv, one for each edge u −− v in G. The axioms are α(v) = [E(v)]l(v)⊕1 for each v ∊ V (see exercise 244), where E(v) = {euv | u −− v} and l(v) is the label of v.

For example, vertex 1 of the graph below is shown as a black dot in order to indicate that l(1) = 1, while the other vertices appear as white dots and are labeled l(2) = ··· = l(6) = 0. The graph and its axioms are

Notice that, when v has d > 0 neighbors in G, the set α(v) consists of 2d−1 clauses of size d. Furthermore, the axioms of α(v) are all satisfied if and only if

If we sum this equation over all vertices v, mod 2, we get 0 on the left, because each edge euv occurs exactly twice (once in E(u) and once in E(v)). But we get 1 on the right. Therefore the clauses α(G) = ∪ vα(v) are unsatisfiable.

1. The axioms α(G) | b and α(G)∣b¯ in this example turn out to be α(G′) and α(G″), where and . Explain what happens in general.

2. Let μ(C) = min{ |V′| | V′ ⊆ V and ∪v∊V,α(v) ⊢ C}, for every clause C involving the variables euv. Show that μ(C) = 1 for every axiom C ∊ α(G). What is μ(∊)?

3. If V′ ⊆ V , let ∂V′ = {euv | u ∊ V′ and v ∉ V′}. Prove that, if ∪v∊V, α(v) ⊢ C and |V′| = μ(C), every variable of ∂V′ appears in C.

4. A nonbipartite cubic Ramanujan graph G on m vertices V has three edges v −− vρ, v −− vσ, v −− vτ touching each vertex, where ρ, σ, and τ are permutations with the following properties: (i) ρ = ρ− and τ = σ−; (ii) G is connected; (iii) If V′ is any subset of s vertices, and if there are t edges between V′ and V \V′, then we have s/(s + t) ≤ (s/m + 8)/9. Prove that w(α(G) ⊢ ∊) > m/78.

▸ 246. [M28] (G. S. Tseytin.) Given a labeled graph G with m edges, n vertices, and N unsatisfiable clauses α(G) as in the previous exercise, explain how to refute those clauses with O(mn + N) steps of extended resolution.

247. [18] Apply variable elimination to just five of the six clauses (112), omitting ‘12¯’.

248. [M20] Formally speaking, SAT is the problem of evaluating the quantified formula

∃x_1 ··· ∃xn−1 ∃xnF(x_1, …, xn−1,xn),
where F is a Boolean function given in CNF as a conjunction of clauses. Explain how to transform the CNF for F into the CNF for F′ in the reduced problem

∃x_1 ··· ∃xn−1F′ (x_1,..., xn−1), F′ (x_1,..., xn−1) = F(x_1,..., xn−1, 0)∨F (x_1,..., xn−1, 1).

249. [18] Apply Algorithm I to (112) using Cook’s Method IA.

250. [25] Since the clauses R′ in (7) are satisfiable, Algorithm I might discover a solution without ever reaching step I4. Try, however, to make the choices in steps I2, I3, and I4 so that the algorithm takes as long as possible to discover a solution.

▸ 251. [30] Show that Algorithm I can prove the unsatisfiability of the anti-maximal-element clauses (99)–(101) by making O(m^3) resolutions, if suitably clairvoyant choices are made in steps I2, I3, and I4.

252. [M26] Can the unsatisfiability of (99)–(101) be proved in polynomial time by repeatedly performing variable elimination and subsumption?

▸ 253. [18] What are the next two clauses learned if decision ‘5’ follows next after (114)?

254. [16] Given the binary clauses {12,1¯3,23¯,2¯4¯,3¯4}, what clause will a CDCL solver learn first if it begins by deciding that 1 is true?

▸ 255. [20] Construct a satisfiability problem with ternary clauses, for which a CDCL solver that is started with decision literals ‘1’, ‘2’, ‘3’ on levels 1, 2, and 3 will learn the clause ‘45’ after a conflict on level 3.

256. [20] How might the clause ‘∗∗’ in Table 3 have been easily learned?

▸ 257. [30] (Niklas Sörensson.) A literal l¯ is said to be redundant, with respect to a given clause c and the current trail, if l is in the trail and either (i) l is defined at level 0, or (ii) l is not a decision literal and every false literal in l’s reason is either in c or (recursively) redundant. (This definition is stronger than the special cases by which (115) reduces to (116), because l¯ itself needn’t belong to c.) If, for example, c=(l¯′∨b¯1∨b¯2∨b¯3∨b¯4), let the reason for b_4be (b4∨b¯1∨a¯1), where the reason for a_1is (a1∨b¯2∨a¯2) and the reason for a_2 is (a2∨b¯1∨b¯3). Then b¯4 is redundant, because ā_2 and ā_1 are redundant.

1. Suppose c=(l¯′∨b¯1∨⋅⋅⋅∨b¯r) is a newly learned clause. Prove that if b¯j∈c is redundant, some other b¯i∈c became false on the same level of the trail as b¯j did.

2. Devise an efficient algorithm that discovers all of the redundant literals b¯j in a given newly learned clause c=(l¯′∨b¯1∨⋅⋅⋅∨b¯r). Hint: Use stamps.

258. [21] A non-decision literal l in Algorithm C’s trail always has a reason Rl = (l_0 ∨ l_1 ∨ ··· ∨ lk−1), where l_0= l and l¯1,..., l¯k−1 precede l in the trail. Furthermore, the algorithm discovered this clause while looking at the watch list of l_1. True or false: l¯2,..., l¯k−1 precede l¯1 in the trail. Hint: Consider Table 3 and its sequel.

259. [M20] Can ACT(j) exceed ACT(k) for values of ρ near 0 or 1, but not for all ρ?

260. [18] Describe in detail step C1’s setting-up of MEM, the watch lists, and the trail.

261. [21] The main loop of Algorithm C is the unit-propagation process of steps C3 and C4. Describe the low-level details of link adjustment, etc., to be done in those steps.

262. [20] What low-level operations underlie changes to the heap in steps C6–C8?

263. [21] Write out the gory details by which step C7 constructs a new clause and step C9 puts it into the data structures of Algorithm C.

264. [20] Suggest a way by which Algorithm C could indicate progress by displaying “move codes” analogous to those of Algorithms A, B, D, and L. (See exercise 142.)

265. [21] Describe several circumstances in which the watched literals l_0 and/or l_1 of a clause c actually become false during the execution of Algorithm C.

266. [20] In order to keep from getting into a rut, CDCL solvers are often designed to make decisions at random, with a small probability p (say p = .02), instead of always choosing a variable of maximum activity. How would this policy change step C6?

▸ 267. [25] Instances of SAT often contain numerous binary clauses, which are handled efficiently by the unit-propagation loop (62) of Algorithm L but not by the corresponding loop in step C3 of Algorithm C. (The technique of watched literals is great for long clauses, but it is comparatively cumbersome for short ones.) What additional data structures will speed up Algorithm C’s inner loop, when binary clauses are abundant?

268. [21] When Algorithm C makes a literal false at level 0 of the trail, we can remove it from all of the clauses. Such updating might take a long time, if we did it “eagerly”; but there’s a lazy way out: We can delete a permanently false literal if we happen to encounter it in step C3 while looking for a new literal to watch (see exercise 261).

Explain how to adapt the MEM data structure conventions so that such deletions can be done in situ, without copying clauses from one location into another.

269. [23] Suppose Algorithm C reaches a conflict at level d of the trail, after having chosen the decision literals u_1, u_2, ... , ud. Then the “trivial clause” (l¯′∨u¯1∨⋅⋅⋅∨u¯d′) must be true if the given clauses are satisfiable, where l′and d′ are defined in step C7.

1. Show that, if we start with the clause (l¯′∨b¯1∨⋅⋅⋅∨b¯r) that is obtained in step C7 and then resolve it somehow with zero or more known clauses, we can always reach a clause that subsumes the trivial clause.

2. Sometimes, as in (115), the clause that is slated to be learned in step C9 is much longer than the trivial clause. Construct an example in which d = 3, d′= 1, and r = 10, yet none of b¯1,...,b¯r are redundant in the sense of exercise 257.

3. Suggest a way to improve Algorithm C accordingly.

270. [25] (On-the-fly subsumption.) The intermediate clauses that arise in step C7, immediately after resolving with a reason Rl, occasionally turn out to be equal to the shorter clause Rl \ l. In such cases we have an opportunity to strengthen that clause by deleting l from it, thus making it potentially more useful in the future.

1. Construct an example where two clauses can each be subsumed in this way while resolving a single conflict. The subsumed clauses should both contain two literals assigned at the current level in the trail, as well as one literal from a lower level.

2. Show that it’s easy to recognize such opportunities, and to strengthen such clauses efficiently, by modifying the steps of answer 263.

▸ 271. [25] The sequence of learned clauses C_1, C_2, ... often includes cases where Ci subsumes its immediate predecessor, Ci−1. In such cases we might as well discard Ci−1, which appears at the very end of MEM, and store Ci in its place, unless Ci−1 is still in use as a reason for some literal on the trail. (For example, more than 8,600 of the 52,000 clauses typically learned from waerden(3, 10; 97) by Algorithm C can be discarded in this way. Such discards are different from the on-the-fly subsumptions considered in exercise 270, because the subsumed Ci−1 includes only one literal from its original conflict level; furthermore, learned clauses have usually been significantly simplified by the procedure of exercise 257, unless they’re trivial.)

Design an efficient way to discover when Ci−1 can be safely discarded.

272. [30] Experiment with the following idea: The clauses of waerden (j, k; n) are symmetrical under reflection, in the sense that they remain unchanged overall if we replace xk by xkR=xn+1−k for 1 ≤ k ≤ n. Therefore, whenever Algorithm C learns a clause C=(l¯′∨b¯1∨⋅⋅⋅∨b¯r), it is also entitled to learn the reflected clause CR=(l¯′R∨b¯1R∨⋅⋅⋅∨b¯rR).

273. [27] A clause C that is learned from waerden (j, k; n) is valid also with respect to waerden(j, k; n′) when n′ > n; and so are the clauses C + i that are obtained by adding i to each literal of C, for 1 ≤ i ≤ n′ −n. For example, the fact that ‘35’ follows from waerden (3, 3; 7) allows us to add the clauses 35, 46, 57 to waerden (3, 3; 9).

1. Exploit this idea to speed up the calculation of van der Waerden numbers.

2. Explain how to apply it also to bounded model checking.

274. [35] Algorithm C sets the “reason” for a literal l as soon as it notices a clause that forces l to be true. Later on, other clauses that force l are often encountered, in practice; but Algorithm C ignores them, even though one of them might be a “better reason.” (For example, another forcing clause might be significantly shorter.) Explore a modification of Algorithm C that tries to improve the reasons of non-decision literals.

▸ 275. [22] Adapt Algorithm C to the problem of finding the lexicographically smallest solution to a satisfiability problem, by incorporating the ideas of exercise 109.

276. [M15] True or false: If F is a family of clauses and L is a set of strictly distinct literals, then F ∧ L ⊢1 ∊ if and only if (F |L) ⊢1 ∊.

277. [M18] If (C_1,..., Ct) is a certificate of unsatisfiability for F , and if all clauses of F have length ≥ 2, prove that some Ci is a unit clause.

278. [22] Find a six-step certificate of unsatisfiability for waerden (3, 3; 9).

279. [M20] True or false: Every unsatisfiable 2SAT problem has a certificate ‘(l, ∊)’.

▸ 280. [M26] The problem cook (j, k) consists of all (nj) positive j-clauses and all (nk)negative k-clauses on {1,..., n}, where n = j + k − 1. For example, cook (2, 3) is

{12,13,14,23,24,34,1¯2¯3¯,1¯2¯4¯,1¯3¯4¯,2¯3¯4¯}

1. Why are these clauses obviously unsatisfiable?

2. Find a totally positive certificate for cook (j, k), of length (n−1j−1).

3. Prove in fact that Algorithm C always learns exactly (n−1j−1) clauses when it proves the unsatisfiability of cook(j, k), if Mp= Mf= ∞ (no purging or flushing).

281. [21] Construct a certificate of unsatisfiability that refutes (99), (100), (101).

▸ 282. [M33] Construct a certificate of unsatisfiability for the clauses fsnark(q) of exercise 176 when q ≥ 3 is odd, using O(q) clauses, all having length ≤ 4. Hint: Include the clauses (āj,p∨ēj,p), (a¯j,p∨f¯j,p), (e¯j,p∨f¯j,p), and (aj,p∨ej,p∨fj,p) for 1 ≤ j ≤ q, 1 ≤ p ≤ 3.

283. [HM46] Does Algorithm C solve the flower snark problem in linear time? More precisely, let pq(M) be the probability that the algorithm refutes fsnark(q) while making at most M references to MEM. Is there a constant N such that pq(Nq)>12 for all q?

284. [23] Given F and (C_1,..., Ct), a certificate-checking program tests condition (119) by verifying that F and clauses C_1, ... , Ci−1 will force a conflict when they are augmented by the unit literals of C¯i. While doing this, it can mark each clause of F ∪ {C_1,..., Ci−1} that was reduced to a unit during the forcing process; then the truth of Ci does not depend on the truth of any unmarked clause.

In practice, many clauses of F are never marked at all, hence F will remain unsatisfiable even if we leave them out. Furthermore, many clauses Ci are not marked during the verification of any of their successors, {Ci+1,..., Ct}; such clauses Ci needn’t be verified, nor need we mark any of the clauses on which they depend.

Therefore we can save work by checking the certificate backwards: Start by marking the final clause Ct, which is ∊ and always needs to be verified. Then, for i = t, t − 1, ... , check Ci only if it has been marked.

The unit propagations can all be done without recording the “reason” Rl that has caused any literal l to be forced. In practice, however, many of the forced literals don’t actually contribute to the conflicts that arise, and we don’t want to mark any clauses that aren’t really involved.

Explain how to use reasons, as in Algorithm C, so that clauses are marked by the verifier only if they actually participate in the proof of a marked clause Ci.

285. [19] Using the data in Fig. 93, the text observes that Eq. (124) gives j = 95, sj = 3081, and mj = 59 when α=1516. What are j, sj, and mj when (a) α=916? (b) α=12? (c) α=716? Also compare the effectiveness of different α’s by computing the number bj of “black” clauses (those with 0 < RANGE(c) < j that proved to be useful).

286. [M24] What choice of signatures-to-keep in Fig. 93 is optimum, in the sense that it maximizes Σ bpqxpq subject to the conditions Σ apqxpq ≤ 3114, xpq ∊ {0, 1}, and xpq ≥ xp′q′ for 1 ≤ p ≤ p′ ≤ 7, 0 ≤ q ≤ q′ ≤ 8? Here apq and bpq are the areas of the gray and black clauses that have signature (p, q), as given by the matrices in the text. [This is a special case of the “knapsack problem with a partial ordering.”]

287. [25] What changes to Algorithm C are necessary to make it do a “full run,” and later to learn from all of the conflicts that arose during that run?

288. [28] Spell out the details of computing RANGE scores and then compressing the database of learned clauses, during a round of purging.

289. [M20] Assume that the kth round of purging begins with yk clauses in memory after kΔ+(k2)δ clauses have been learned, and that purging removes 12yk of those clauses. Find a closed formula for yk as a function of k.

290. [17] Explain how to find xk, the unassigned variable of maximum activity that is used for flushing literals. Hint: It’s in the HEAP array.

291. [20] In the text’s hypothetical scenario about flushing Table 3 back to level 15, why will 49 soon appear on the trail, instead of 49¯?

292. [M21] How large can AGILITY get after repeatedly executing (127)?

293. [21] Spell out the details of updating Mf to M + Δf when deciding whether or not to flush. Also compute the agility threshold that’s specified in Table 4. Hint: See (131).

294. [HM21] For each binary vector α = x_1x_2x_3x_4, find the generating function gα(z)=Σj=0∞pα,jzj , where pα,j is the probability that Algorithm P will solve the seven clauses of (7) after making exactly j flips, given the initial values α in step P1. Deduce the mean and variance of the number of steps needed to find a solution.

295. [M23] Algorithm P often finds solutions much more quickly than predicted by Corollary W. But show that some 3SAT clauses will indeed require Ω((4/3)n) trials.

296. [HM20] Complete the proof of Theorem U by (approximately) maximizing the quantity f(p, q) in (129). Hint: Consider f(p + 1, q)/f(p, q).

▸ 297. [HM26] (Emo Welzl.) Let Gq(z) = ΣpCp,p+q−1(z/3)p+q(2z/3)p be the generating function for stopping time t = 2p + q when Y_0= q in the proof of Theorem U.

1. Find a closed form for Gq(z), using formulas from Section 7.2.1.6.

2. Explain why Gq(1) is less than 1.

3. Evaluate and interpret the quantity Gq′(1)/Gq(1).

4. Use Markov’s inequality to bound the probability that Yt = 0 for some t ≤ N.

5. Show that Corollary W follows from this analysis.

298. [HM22] Generalize Theorem U and Corollary W to the case where each clause has at most k literals, where k ≥ 3.

299. [HM23] Continuing the previous exercise, investigate the case k = 2.

▸ 300. [25] Modify Algorithm P so that it can be implemented with bitwise operations, thereby running (say) 64 independent trials simultaneously.

▸ 301. [25] Discuss implementing the algorithm of exercise 300 efficiently on MMIX.

302. [26] Expand the text’s high-level description of steps W4 and W5, by providing low-level details about exactly what the computer should do.

303. [HM20] Solve exercise 294 with Algorithm W in place of Algorithm P.

304. [HM34] Consider the 2SAT problem with n(n − 1) clauses (x¯j∨xk) for all j ≠ k. Find the generating functions for the number of flips taken by Algorithms P and W. Hint: Exercises 1.2.6–68 and MPR–105 are helpful for finding the exact formulas.

▸ 305. [HM29] Add one more clause, (x¯1∨x¯2), to the previous exercise and find the resulting generating functions when n = 4. What happens when p = 0 in Algorithm W?

▸ 306. [HM32] (Luby, Sinclair, and Zuckerman, 1993.) Consider a “Las Vegas algorithm” that succeeds or fails; it succeeds at step t with probability pt, and fails with probability p∞ < 1. Let qt = p_1 + p_2 + ··· + pt and Et = p_1 + 2p_2 + ··· + tpt; also let E∞ = ∞ if p∞ > 0, otherwise E∞=∑ttpt. (The latter sum might be ∞.)

1. Suppose we abort the algorithm and restart it again, whenever the first N steps have not succeeded. Show that if qN > 0, this strategy will succeed after performing an average of l(N) < ∞ steps. What is l(N)?

2. Compute l(N) when pm=mn, p∞=n−mn, otherwise pt = 0, where 1 ≤ m ≤ n.

3. Given the uniform distribution, pt=1n for 1 ≤ t ≤ n, what is l(N)?

4. Find all probability distributions such that l(N) = l(1) for all N ≥ 1.

5. Find all probability distributions such that l(N) = l(n) for all N ≥ n.

6. Find all probability distributions such that qn+1 = 1 and l(n) ≤ l(n + 1).

7. Find all probability distributions such that q_3 = 1 and l(1) < l(3) < l(2).

8. Let l = infN≥1 l(N), and let N∗ be the least positive integer such that l(N∗) = l, or ∞ if no such integer exists. Prove that N∗ = ∞ implies l = E∞ < ∞.

9. Find N∗ for the probability distribution pt =[t>n]/((t−n)(t+1−n)), given n ≥ 0.

10. Exhibit a simple example of a probability distribution for which N∗ = ∞.

11. Let L = mint≥1 t/qt. Prove that l ≤ L ≤ 2l − 1.

307. [HM28] Continuing exercise 306, consider a more general strategy defined by an infinite sequence of positive integers (N_1, N_2, ... ): “Set j ← 0; then, while success has not yet been achieved, set j ← j + 1 and run the algorithm with cutoff parameter Nj.”

1. Explain how to compute E X, where X is the number of steps taken before this strategy succeeds.

2. Let Tj = N_1 + ··· + Nj. Prove that E⁢ X=∑j=1∞Pr(Tj−1<X≤Tj)l(Nj), if we have qNj > 0 for all j.

3. Consequently the steady strategy (N∗, N∗, ... ) is best: E X ≥ l(N∗) = l.

4. Given n, exercise 306(b) defines n simple probability distributions p(m) that have l(N∗) = n, but the value of N∗ = m is different in each case. Prove that any sequence (N_1, N_2, ... ) must have E⁢  X>14nHn−12n=14lHl−12l on at least one of those p(m). Hint: Consider the smallest r such that, for each m, the probability is ≥12 that r trial runs suffice; show that ≥ n/(2m) of {N_1, ..., Nr} are ≥ m.

308. [M29] This exercise explores the “reluctant doubling” sequence (130).

1. What is the smallest n such that Sn = 2a, given a ≥ 0?

2. Show that {n | Sn = 1} = {2k + 1 − νk | k ≥ 0}; hence the generating function ∑n z n[Sn = 1] is the infinite product z(1 + z)(1 + z^3)(1 + z^7)(1 + z^15) ....

3. Find similar expressions for {n | Sn = 2a} and ∑n z n[Sn = 2a].

4. Let Σ(a,b,k)=∑n=1r(a,b,k)Sn, where Sr(a,b,k) is the 2bkth occurrence of 2a in 〈Sn〉. For example, ∑(1, 0, 3) = S_1 + ··· + S_10 = 16. Evaluate ∑(a, b, 1) in closed form.

5. Show that ∑(a, b, k+1) − ∑(a, b, k) ≤ (a + b +2k − 1)2a+b, for all k ≥ 1.

6. Given any probability distribution as in exercise 306(k), let a = ⌈lg t⌉ and b = ⌈lg 1/qt⌉, where t/qt = L; thus L ≤ 2a+b < 4L. Prove that if the strategy of exercise 307 is used with Nj = Sj, we have

E⁢  X≤Σ(a,b,1)+Σk≥1Qk(Σ(a,b,k+1)−Σ(a,b,k)),where Q=(1−q2a)2b.

7. Therefore 〈Sn〉 gives E X < 13l lg l + 49l, for every probability distribution.

309. [20] Exercise 293 explains how to use the reluctant doubling sequence with Algorithm C. Is Algorithm C a Las Vegas algorithm?

310. [M25] Explain how to compute the “reluctant Fibonacci sequence”

1, 1, 2, 1, 2, 3, 1, 1, 2, 3, 5, 1, 1, 2, 1, 2, 3, 5, 8, 1, 1, 2, 1, 2, 3, 1, 1, 2, 3, 5, 8, 13, 1, ...,
which is somewhat like (130) and useful as in exercise 308, but its elements are Fibonacci numbers instead of powers of 2.

311. [21] Compute approximate values of E X for the 100 probability distributions of exercise 306(b) when n = l = 100, using the method of exercise 307 with the sequences 〈Sn〉 of exercise 308 and 〈Sn′〉 of exercise 310. Also consider the more easily generated “ruler doubling” sequence 〈Rn〉, where Rn = n & −n = 2ρn. Which sequence is best?

312. [HM24] Let T (m, n) = E X when the reluctant doubling method is applied to the probability distribution defined in exercise 306(b). Express T (m, n) in terms of the generating functions in exercise 308(c).

▸ 313. [22] Algorithm W always flips a cost-free literal if one is present in Cj, without considering its parameter p. Show that such a flip always decreases the number of unsatisfied clauses, r; but it might increase the distance from x to the nearest solution.

▸ 314. [36] (H. H. Hoos, 1998.) If the given clauses are satisfiable, and if p > 0, can there be an initial x for which Algorithm W always loops forever?

315. [M18] What value of p is appropriate in Theorem J when d = 1?

316. [HM20] Is Theorem J a consequence of Theorem L?

▸ 317. [M26] Let α(G)=Pr(A1¯∩⋅⋅⋅∩A¯m) under the assumptions of (133), when pi = p = (d − 1)d−1/dd for 1 ≤ i ≤ m and every vertex of G has degree at most d > 1. Prove, by induction on m, that α(G) > 0 and that α(G)>d−1dα(G\v) when v has degree < d.

318. [HM27] (J. B. Shearer.) Prove that Theorem J is the best possible result of its kind: If p > (d − 1)d−1/dd and d > 1, there is a graph G of maximum degree d for which (p,...,p) ∉ R(G). Hint: Consider complete t-ary trees, where t = d − 1.

319. [HM20] Show that pde < 1 implies p ≤ (d − 1)d−1/dd.

320. [M24] Given a lopsidependency graph G, the occurrence threshold ρ(G) is the smallest value p such that it’s sometimes impossible to avoid all events when each event occurs with probability p. For example, the Möbius polynomial for the path P_3 is 1−p_1−p_2−p_3 +p_1p_3; so the occurrence threshold is ϕ−2, the least p with 1−3p+p^2 ≤ 0.

1. Prove that the occurrence threshold for Pm is 1/(4⁢ cos2πm+2).

2. What is the occurrence threshold for the cycle graph Cm?

321. [M24] Suppose each of four random events A, B, C, D occurs with probability p, where {A, C} and {B, D} are independent. According to exercise 320(b) with m = 4, there’s a joint distribution of (A, B, C, D) such that at least one of the events always occurs, whenever p≥(2−2)/2≈0.293. Exhibit such a distribution when p = 3/10.

▸ 322. [HM35] (K. Kolipaka and M. Szegedy, 2011.) Surprisingly, the previous exercise cannot be solved in the setting of Algorithm M! Suppose we have independent random variables (W, X, Y, Z) such that A depends on W and X, B depends on X and Y, C depends on Y and Z, D depends on Z and W. Here W equals j with probability wj for all integers j; X, Y, and Z are similar. This exercise will prove that the constraint A¯∩B¯∩C¯∩D¯ is always satisfiable, even when p is as large as 0.333.

1. Express the probability Pr(A¯∩B¯∩C¯∩D¯) in a convenient way.

2. Suppose there’s a distribution of W, X, Y, Z with Pr(A) = Pr(B) = Pr(C) = Pr(D) = p and Pr(A¯∩B¯∩C¯∩D¯)=0. Show that there are ten values such that

0≤a,b,c,d,a′,b′,c′,d′≤1,0<μ,ν<1,μa+(1−μ)a′≤p,μb+(1−μ)b′≤p,νc+(1−ν)c′≤p,vd+(1−ν)d′≤p,a+d≥1or⁢ b+c≥1,a+d′≥1⁢ or⁢ b+c′≥1,a′+d≥1⁢ or⁢  b′+c≥1,a′+d′≥1⁢  or⁢  b′+c′≥1.

3. Find all solutions to those constraints when p = 1/3.

4. Convert those solutions to distributions that have Pr(A¯∩B¯∩C¯∩D¯)=0.

323. [10] What trace precedes ccb in the list (135)?

▸ 324. [22] Given a trace α = x_1x_2 ... xn for a graph G, explain how to find all strings β that are equivalent to α, using Algorithm 7.2.1.2V. How many strings yield (136)?

▸ 325. [20] An acyclic orientation of a graph G is an assignment of directions to each of its edges so that the resulting digraph has no oriented cycles. Show that the number of traces for G that are permutations of the vertices (with each vertex appearing exactly once in the trace) is the number of acyclic orientations of G.

326. [20] True or false: If α and β are traces with α = β, then αR = βR. (See (137).)

▸ 327. [22] Design an algorithm to multiply two traces α and β, when clashing is defined by territory sets T(a) in some universe U. Assume that U is small (say |U| ≤ 64), so that bitwise operations can be used to represent the territories.

328. [20] Continuing exercise 327, design an algorithm that computes α/β. More precisely, if β is a right factor of α, in the sense that α = γβ for some trace γ, your algorithm should compute γ; otherwise it should report that β is not a right factor.

329. [21] Similarly, design an algorithm that either computes α/β or reports that α isn’t a left factor of β.

▸ 330. [21] Given any graph G, explain how to define territory sets T (a) for its vertices a in such a way that we have a = b or a −−− b if and only if T (a) ∩ T(b) ≠ ∅. (Thus traces can always be modeled by empilements of pieces.) Under what circumstances is it possible to do this with |T(a)| = 2 for all a, as in the text’s example (136)?

331. [M20] What happens if the right-hand side of (139) is expanded without allowing any of the variables to commute with each other?

332. [20] When a trace is represented by its lexicographically smallest string, no letter in that representative string is followed by a smaller letter with which it commutes. (For example, no c is followed by a in (135), because we could get an equivalent smaller string by changing ca to ac.)

Conversely, given any ordered set of letters, some of which commute, consider all strings having no letter followed by a smaller letter with which it commutes. Is every such string the lexicographically smallest of its trace?

▸ 333. [M20] (Carlitz, Scoville, and Vaughan, 1976.) Let D be a digraph on {1, ...,m}, and let A be the set of all strings aj1 ...ajn such that ji → ji+1 in D for 1 ≤ i < n. Similarly let B be the set of all strings aj1 ...ajn such that ji ↛ ji+1 for 1 ≤ i < n. Prove that

Σα∈Aα=1/Σβ∈B(−1)|β| β=Σk≥0(1−Σβ∈B(−1)|β|⁢ β)k
is an identity in the noncommutative variables {a_1, ...,am}. (For example, we have

1+a+b+ab+ba+aba+bab+⋅⋅⋅=Σk≥0(a+b−aa−bb+aaa+bbb−⋅⋅⋅)k
in the case m = 2, 1 ↛ 1, 1 → 2, 2 ↛ 1, 2 ↛ 2.)

▸ 334. [25] Design an algorithm to generate all traces of length n that correspond to a given graph on the alphabet {1, ...,m}, representing each trace by its lexicographically smallest string.

335. [HM26] If the vertices of G can be ordered in such a way that x < y < z and x―y and y―z implies x―z, show that the Möbius series MG can be expressed as a determinant. For example,

▸ 336. [M20] If graphs G and H on distinct vertices have the Möbius series MG and MH, what are the Möbius series for (a) G ⊕ H and (b) G −−− H?

337. [M20] Suppose we obtain the graph G′ from G by substituting a clique of vertices {a_1, ...,ak} for some vertex a, then including edges from aj to each neighbor of a for 1 ≤ j ≤ k. Describe the relation between MG′ and MG.

338. [M21] Prove Viennot’s general identity (144) for source-constrained traces.

▸ 339. [HM26] (G. Viennot.) This exercise explores factorization of traces into pyramids.

1. Each letter xj of a given trace α = x_1 ...xn lies at the top of a unique pyramid βj such that βj is a left factor of α. For example, in the trace bcebafdc of (136), the pyramids β_1, ... , β_8 are respectively b, bc, e, bcb, bcba, ef, bced, and bcebdc. Explain intuitively how to find these pyramidal left factors from α’s empilement.

2. A labeled trace is an assignment of distinct numbers to the letters of a trace; for example, abca might become a_4b_7c_6a_3. A labeled pyramid is the special case when the pyramid’s top element is required to have the smallest label. Prove that every labeled trace is uniquely factorizable into labeled pyramids whose topmost labels are in ascending order. (For example, b_6c_2e_4b_7a_8f_5d_1c_3 = b_6c_2e_4d_1 · b_7a_8c_3 · f_5.)

3. Suppose there are tn traces of length n, and pn pyramids. Then there are Tn = n! tn labeled traces and Pn = (n − 1)! pn labeled pyramids (because only the relative order of the labels is significant). Letting T(z) = ∑n≥0 Tnzn/n! and P(z) = ∑n≥1 Pnzn/n!, prove that the number of labeled traces of length n whose factorization in part (b) has exactly l pyramids is n![zn] P(z)l/l!.

4. Consequently T (z) = eP(z).

5. Therefore (and this is the punch line!) ln MG(z) = ∑n≥1 pnzn/n.

▸ 340. [M20] If we assign a weight w(σ) to every cyclic permutation σ, then every permutation π has a weight w(π) that is the product of the weights of its cycles. For example, if π=(12345673142765)=(1⁢  3⁢ ⁢4⁢ ⁢2)(5⁢ 7)(6) then w(π) = w((1 3 4 2))w((5 7))w((6)).

The permutation polynomial of a set S is the sum of w(π) over all permutations of S. Given any n × n matrix A = (aij), show that it’s possible to define appropriate cycle weights so that the permutation polynomial of {1, ...,n} is the determinant of A.

341. [M25] The involution polynomial of a set S is the special case of the permutation polynomial when the cycle weights have the form wjjx for the 1-cycle (j) and −wij for the 2-cycle (ij), otherwise w(σ) = 0. For example, the involution polynomial of {1, 2, 3, 4} is w_11w_22w_33w_44x^4 − w_11w_22w_34x^2 − w_11w_23w_44x^2 − w_11w_24w_33x^2 − w_12w_33w_44x^2 − w_13w_22w_44x^2 − w_14w_22w_33x^2 + w_12w_34 + w_13w_24 + w_14w_23.

Prove that, if wij > 0 for 1 ≤ i ≤ j ≤ n, the involution polynomial of {1, ...,n} has n distinct real roots. Hint: Show also that, if the roots for {1, ...,n − 1} are q_1 < ··· < qn−1, then the roots rk for {1, ...,n} satisfy r_1 < q_1 < r_2 < ··· < qn−1 < rn.

342. [HM25] (Cartier and Foata, 1969.) Let Gn be the graph whose vertices are the ∑k=1n(nk)(k−1) cyclic permutations of subsets of {1, ...,n}, with σ −−− τ when σ and τ intersect. For example, the vertices of G_3 are (1), (2), (3), (12), (13), (23), (123), (132); and they’re mutually adjacent except that (1)―(2), (1)―(3), (1)―(23), (2)―(3), (2)―(13), (12)―(3). Find a beautiful relation between MGn and the characteristic polynomial of an n × n matrix.

▸ 343. [M25] If G is any cograph, show that (p_1, ...,pm) ∊ 𝓡(G) if and only if we have MG(p_1, ...,pm) > 0. Exhibit a non-cograph for which the latter statement is not true.

344. [M33] Given a graph G as in Theorem S, let B_1, ... , Bm have the joint probability distribution of exercise MPR–31, with πI = 0 whenever I contains distinct vertices {i, j} with i −−− j, otherwise πI = ∏i∊I pi.

1. Show that this distribution is legal (see exercise MPR–32) if (p_1, ...,pm) ∊ R(G).

2. Show that this “extreme distribution” also satisfies condition (147).

3. Let β(G)=Pr(B¯1∩⋅⋅⋅∩B¯m). If J ⊆ {1, ...,m}, express β(G|J) in terms of MG.

4. Defining α(G) as in exercise 317, with events Aj satisfying (133) and probabilities (p_1, ...,pm) ∊ 𝓡(G), show that α(G | J) ≥ β(G | J) for all J ⊆ {1, ...,m}.

5. If pi satisfies (134), show that β(G|J) ≥ ∏j∊J (1 − θj).

345. [M30] Construct unavoidable events that satisfy (147) when (p_1, ...,pm) ∈ 𝓡(G).

▸ 346. [HM28] Write (142) as MG = MG\a(1 − aKa,G) where Ka,G = MG\a∗ /MG\a.

1. If (p_1, ...,pm) ∊ 𝓡(G), prove that Ka,G is monotonic in all of its parameters: It does not increase if any of p_1, ... , pm are decreased.

2. Exploit this fact to design an algorithm that computes MG(p_1, ...,pm) and decides whether or not (p_1, ...,pm) ∊ 𝓡(G), given a graph G and probabilities (p_1, ...,pm). Illustrate your algorithm on the graph G = P_3 □ P_2 of exercise 335.

▸ 347. [M28] A graph is called chordal when it has no induced cycle Ck for k > 3. Equivalently (see Section 7.4.2), a graph is chordal if and only if its edges can be defined by territory sets T(a) that induce connected subgraphs of some tree. For example, interval graphs and forests are chordal.

1. Say that a graph is tree-ordered if its vertices can be arranged as nodes of a forest in such a way that
a―b⁢   implies⁢  a≻b⁢  or⁢   b≻a;a≻b≻c⁢   and⁢  a—c⁢  implies⁢   a—b.(*)

(Here ‘a ≻ b’ means that a is a proper ancestor of b in the forest.) Prove that every tree-ordered graph is chordal.

2. Conversely, show that every chordal graph can be tree-ordered.

3. Show that the algorithm in the previous exercise becomes quite simple when it is applied to a tree-ordered graph, if a is eliminated before b whenever a ≻ b.

4. Consequently Theorem L can be substantially strengthened when G is a chordal graph: When G is tree-ordered by ≻, the probability vector (p_1, ...,pm) is in 𝓡(G) if and only if there are numbers 0 ≤ θ_1, ...,θm < 1 such that
pi=θi⁢  Πi—j⁢ in G, i≻j(1−θj).

348. [HM26] (A. Pringsheim, 1894.) Show that any power series f(z)=Σn=0∞anzn with an ≥ 0 and radius of convergence ρ, where 0 < ρ < ∞, has a singularity at z = ρ.

▸ 349. [M24] Analyze Algorithm M exactly in the two examples considered in the text (see (150)): For each binary vector x = x_1 ...x_7, compute the generating function gx(z)=Σt⁢ px,tzt, where px,t is the probability that step M3 will be executed exactly t times after step M1 produces x. Assume that step M2 always chooses the smallest possible value of j. (Thus the ‘Case 2’ scenario in (150) will never occur.)

What are the mean and variance of the running times, in (i) Case 1? (ii) Case 2?

350. [HM26] (W. Pegden.) Suppose Algorithm M is applied to the m = n + 1 events

Aj = xj for 1 ≤ j ≤ n; Am = x_1 ∨···∨ xn.

Thus Am is true whenever any of the other Aj is true; so we could implement step M2 by never setting j ← m. Alternatively, we could decide to set j ← m whenever possible. Let (Ni, Nii, Niii, Niv, Nv) be the number of resamplings performed when parameter ξk of the algorithm is (i) 1/2; (ii) 1/(2n); (iii) 1/2n; (iv) 1/(n + k); (v) 1/(n + k)2.

1. Find the asymptotic mean and variance of each N, if j is never equal to m.

2. Find the asymptotic mean and variance of each N, if j is never less than m.

3. Let G be the graph on {1, ...,n +1} with edges j −−− (n + 1) for 1 ≤ j ≤ n, and let pj = Pr(Aj). For which of the five choices of ξk is (p_1, ...,pn+1) ∊ 𝓡(G)?

▸ 351. [25] The Local Lemma can be applied to the satisfiability problem for m clauses on n variables if we let Aj be the event “Cj is not satisfied.” The dependency graph G then has i −−− j whenever two clauses Ci and Cj share at least one common variable. If, say, Ci is (x3∨x¯5∨x6), then (133) holds whenever pi ≥ (1 − ξ_3)ξ_5(1 − ξ_6), assuming that each xk is true with probability ξk, independent of the other x’s.

But if, say, Cj is (x¯2∨x3∨x7), condition (133) remains true even if we don’t stipulate that i −−− j. Variable x_3 appears in both clauses, yet the cases when Cj is satisfied are never bad news for Ci. We need to require that i −−− j in condition (133) only when Ci and Cj are “resolvable” clauses, namely when some variable occurs positively in one and negatively in the other.

Extend this reasoning to the general setting of Algorithm M, where we have arbitrary events Aj that depend on variables Ξj: Define a lopsidependency graph G for which (133) holds even though we might have i―j in some cases when Ξi ∩ Ξj ≠ ∅.

352. [M21] Show that Ej ≤ θj/(1 − θj) in (152), when (134) holds.

353. [M21] Consider Case 1 and Case 2 of Algorithm M as illustrated in (150).

1. How many solutions x_1 ...xn are possible? (Generalize from n = 7 to any n.)

2. How many solutions are predicted by Theorem S?

3. Show that in Case 2 the lopsidependency graph is much smaller than the dependency graph. How many solutions are predicted when the smaller graph is used?

354. [HM20] Show that the expected number E N of resampling steps in Algorithm M is at most −MG*′ (1)/MG*(1).

355. [HM21] In (152), prove that Ej ≤ 1/δ when (p_1, ...,pm) has positive slack δ. Hint: Consider replacing pj by pj + δpj.

▸ 356. [M33] (The Clique Local Lemma.) Let G be a graph on {1, ...,m}, and let G | U_1, ... , G | Ut be cliques that cover all the edges of G. Assign numbers θij ≥ 0 to the vertices of each Uj, such that ∑j = ∑i∊Uj θij < 1. Assume that

Pr(Ai)=pi≤θij⁢   Πk≠j,i∈Uk(1+θik−Σk)whenever⁢  1≤i≤m⁢        and⁢  i⁢  ∈Uj.⁢

1. Prove that (p_1, ...,pm) ∊ 𝓡(G). Hint: Letting A¯S denote ∩i∈S  A¯i, show that

Pr(Ai| A¯S)≤θijwhenever⁢1≤i≤m⁢  and⁢  i∈Uj⁢   and⁢   S∩Uj=∅.

2. Also Ei in (152) is at most mini−−−j> in G θij/(1 − ∑j). (See Theorems M and S.)

3. Improve Theorem L by showing that, if 0≤θj<12, then (p_1, ...,pm)∊R(G) when
pi=θi(Πi―j⁢ in G(1−θj))/maxi―j⁢ in G(1−θj).

▸ 357. [M20] Let x=πv¯ and y = πv in (155), and suppose the field of variable v is (p, q, r). Express x and y as functions of p, q, and r.

358. [M20] Continuing exercise 357, prove that r = max(p, q, r) if and only if x, y≥12.

359. [20] Equations (156) and (157) should actually have been written
γl→C=(1−πl¯) (1−ηl) Πl∈C′≠C (1−ηC→l)πl¯+(1−πl¯) (1−ηl) Πl∈C′≠C (1−ηC→l)and⁢       ηC→l′=ΠC∍l′≠lγl′→C,
to avoid division by zero. Suggest an efficient way to implement these calculations.

360. [M23] Find all fixed points of the seven-clause system illustrated in (159), given that π1=π2¯=π4¯=1. Assume also that ηlηl¯=0 for all l.

▸ 361. [M22] Describe all fixed points ηC→l=ηC→l′ of the equations (154), (156), (157), for which each ηC→l and each ηl is either 0 or 1.

362. [20] Spell out the computations needed to finish Algorithm S in step S8.

▸ 363. [M30] (Lattices of partial assignments.) A partial assignment to the variables of a satisfiability problem is called stable (or “valid”) if it is consistent and cannot be extended by unit propagation. In other words, it’s stable if and only if no clause is entirely false, or entirely false except for at most one unassigned literal. Variable xk of a partial assignment is called constrained if it appears in a clause where ±xk is true but all the other literals are false (thus its value has a “reason”).

The 3n partial assignments of an n-variable problem can be represented either as strings x = x_1 ...xn on the alphabet {0, 1, ∗} or as sets L of strictly distinct literals. For example, the string x = ∗1∗01∗ corresponds to the set L={2,4¯,5}. We write x ≺ x′ if x′ is equal to x except that xk′∈{0,1}; equivalently L ≺ L′ if L′ = L ∪ k or L′=L∪k¯. Also x ⊑ x′ if there are t ≥ 0 stable partial assignments x(j) with
x = x(0) ≺ x(1) ≺ ··· ≺ x(t) = x′.

Let p_1, ... , pn, q_1, ... , qn be probabilities, with pk +qk = 1 for 1 ≤ k ≤ n. Define the weight W (x) of a partial assignment to be 0 if x is unstable, otherwise

W(x) = ∏{pk | xk = ⋆}. ∏{qk | xk ≠ ∗ and xk is unconstrained}.

[E. Maneva, E. Mossel, and M. J. Wainwright, in JACM 54 (2007), 17:1–17:41, studied general message-passing algorithms on partial assignments that are distributed with probability proportional to their weights, in the case p_1 = ··· = pn = p, showing that survey propagation (Algorithm S) corresponds to the limit as p → 1.]

1. True or false: The partial assignment specified by the literals currently on the trail in step C5 of Algorithm C is stable.

2. What weights W(x) correspond to the clauses F in (1)?

3. Let x be a stable partial assignment with xk = 1, and let x′ and x″ be obtained from x by setting xk′←0, xk"←∗. True or false: xk is unconstrained in x if and only if (i) x′ is consistent; (ii) x′ is stable; (iii) x″ is stable.

4. If the only clause is 123 = (x_1 ∨ x_2 ∨ x_3), find all sets L such that L⊑{1,2¯,3¯}.

5. What are the weights when there’s only a single clause 123 = (x_1 ∨ x_2 ∨ x_3)?

6. Find clauses such that the sets L with L ⊑ {1, 2, 3, 4, 5} are ∅, {4}, {5}, {1, 4}, {2, 5}, {4, 5}, {1, 4, 5}, {2, 4, 5}, {3, 4, 5}, {1, 3, 4, 5}, {2, 3, 4, 5}, {1, 2, 3, 4, 5}.

7. Let 𝓛 be a family of sets ⊆ {1, ...,n}, closed under intersection, with the property that L ∊ 𝓛 implies L = L(0) ≺ L(1) ≺ ··· ≺ L(t) = {1, ...,n} for some L(j) ∊ 𝓛. (The sets in (f) form one such family, with n = 5.) Construct strict Horn clauses with the property that L ∊ 𝓛 if and only if L ⊑ {1, ...,n}.

8. True or false: If L, L′, L″ are stable and L′ ≺ L, L″ ≺ L, then L′ ∩ L″ is stable.

9. If L′ ⊑ L and L″ ⊑ L, prove that L′ ∩ L″ ⊑ L.

10. Prove that ∑x′⊑x W(x′) = ∏{pk | xk = ∗} whenever x is stable.

▸ 364. [M21] A covering assignment is a stable partial assignment in which every assigned variable is constrained. A core assignment is a covering assignment L that satisfies L ⊑ L′ for some total assignment L′.

1. True or false: The empty partial assignment L = ∅ is always covering.

2. Find all the covering and core assignments of the clauses F in (1).

3. Find all the covering and core assignments of the clauses R′ in (7).

4. Show that every satisfying assignment L′ has a unique core.

5. The satisfying assignments form a graph, if two of them are adjacent when they differ by complementing just one literal. The connected components of this graph are called clusters. Prove that the elements of each cluster have the same core.

6. If L′ and L″ have the same core, do they belong to the same cluster?

365. [M27] Prove that the clauses waerden (3, 3; n) have a nontrivial (i.e., nonempty) covering assignment for all sufficiently large n (although they’re unsatisfiable).

▸ 366. [18] Preprocess the clauses R′ of (7). What erp rules are generated?

▸ 367. [20] Justify the erp rule (161) for elimination by resolution.

368. [16] Show that subsumption and downhill resolution imply unit conditioning: Any preprocessor that does transformations 2 and 4 will also do transformation 1.

▸ 369. [21] (N. Eén and A. Biere.) Suppose l appears only in clauses C_1, ... , Cp and l¯ appears only in clauses C1′,...,Cq′, where we have C_1 = (l∨l_1 ∨···∨ lr) and Cj′=(l¯∨l¯j) for 1 ≤ j ≤ r. Prove that we can eliminate |l| by using the erp rule l¯←(l1∨⋅⋅⋅∨lr) and replacing those p + q clauses by only (p − 2)r + q others, namely

{C1◊Cj′| r<j≤q}∪{Ci◊Cj′| 1<i≤p,1≤j≤r}.

(The case r = 1 is especially important. In many applications — for example in the examples of fault testing, tomography, and the “Life in 4” problem about extending Fig. 78 — more than half of all variable eliminations admit this simplification.)

370. [20] The clauses obtained by resolution might be needlessly complex even when exercise 369 doesn’t apply. For example, suppose that variable x appears only in the clauses (x∨a)∧(x∨a¯∨c)∧(x¯∨b)∧(x¯∨b¯∨c¯). Resolution replaces those four clauses by three others: (a∨b)∧(a∨b¯∨c¯)∧(a¯∨b∨c). Show, however, that only two clauses, both binary, would actually suffice in this particular case.

371. [24] By preprocessing repeatedly with transformations 1–4, and using exercise 369, prove that the 32 clauses (9) of waerden (3, 3; 9) are unsatisfiable.

372. [30] Find a “small” set of clauses that cannot by solved entirely via transformations 1–4 and the use of exercise 369.

373. [25] The answer to exercise 228 defines 2m+∑j=1m(j−1)2≈m3/3 clauses in m^2 variables that suffice to refute the anti-maximal-element axioms of (99)–(101). Algorithm L needs exponential time to handle these clauses, according to Theorem R; and experiments show that they are bad news for Algorithm C too. Show, however, that preprocessing with transformations 1–4 will rapidly prove them unsatisfiable.

▸ 374. [32] Design data structures for the efficient representation of clauses within a SAT preprocessor. Also design algorithms that (a) resolve clauses C and C′ with respect to a variable x; (b) find all clauses C′ that are subsumed by a given clause C; (c) find all clauses C′ that are self-subsumed by a given clause C and a literal x¯∈C.

375. [21] Given |l|, how can one test efficiently whether or not the special situation in exercise 369 applies, using (and slightly extending) the data structures of exercise 374?

▸ 376. [36] After a preprocessor has found a transformation that reduces the current set of clauses, it is supposed to try again and look for further simplifications. (See (160).) Suggest methods that will avoid unnecessary repetition of previous work, by using (and slightly extending) the data structures of exercise 374.

377. [22] (V. Vassilevska Williams.) If G is a graph with n vertices and m edges, construct a 2SAT problem F with 3n variables and 6m clauses, such that G contains a triangle (a 3-clique) if and only if F has a failed literal.

378. [20] (Blocked clause elimination.) Clause C = (l ∨ l_1 ∨ ··· ∨ lq) is said to be blocked by the literal l if every clause that contains l¯ also contains either l¯1 or ··· or l¯q. Exercise 161(b) proves that clause C can be removed without making an unsatisfiable problem satisfiable. Show that this transformation requires an erp rule, even though it doesn’t eliminate any of the variables. What erp rule works?

▸ 379. [20] (Blocked self-subsumption.) Consider the clause (a ∨ b ∨ c ∨ d), and suppose that every clause containing ā but not b¯ nor c¯ also contains d. Show that we can then shorten the clause to (a ∨ b ∨ c) without affecting satisfiability. Is an erp rule needed?

380. [21] Sometimes we can use self-subsumption backwards, for example by weakening the clause (l_1∨l_2 ∨l_3) to (l_1 ∨···∨lk) if each intermediate replacement of (l_1∨· · ·∨lj) by (l_1∨···∨lj−1) is justifiable for 3 < j ≤ k. Then, if we’re lucky, the clause (l_1∨· · ·∨lk) is weak enough to be eliminated; in such cases we are allowed to eliminate (l_1 ∨ l_2 ∨ l_3).

1. Show that (a ∨ b ∨ c) can be eliminated if it is accompanied by the additional clauses (a∨b∨d¯), (a ∨ d ∨ e), (b∨d∨e¯).

2. Show that (a ∨ b ∨ c) can also be eliminated when accompanied by (a∨b∨d¯), (a∨c¯∨d¯), (b∨d∨e¯), (b∨c¯∨e¯), provided that no other clauses contain c¯.

3. What erp rules, if any, are needed for those eliminations?

381. [22] Combining exercises 379 and 380, show that any one of the clauses in

(x¯1∨x2)∧(x¯2∨x3)∧⋅⋅⋅∧(x¯n−1∨xn)∧(x¯n∨x1)

can be removed if there are no other clauses with negative literals. State the erp rules.

382. [30] Although the techniques in the preceding exercises are computationally difficult to apply, show that a lookahead forest based on the dependency digraph can be used to discover some of those simplifications efficiently.

▸ 383. [23] (Inprocessing.) A SAT solver can partition its database of current clauses into two parts, the “hard” clauses Φ and the “soft” clauses Ψ. Initially Ψ is empty, while Φ is F , the set of all input clauses. Four kinds of changes are subsequently allowed:

- Learning. We can append a new soft clause C, provided that Φ ∪ Ψ ∪ C is satisfiable whenever Φ ∪ Ψ is satisfiable.

- Forgetting. We can discard (purge) any soft clause.

- Hardening. We can reclassify any soft clause and call it hard.

- Softening. We can reclassify any hard clause C and call it soft, provided that Φ is satisfiable whenever Φ \ C is satisfiable. In this case we also should output any necessary erp rules, which change the settings of variables in such a way that any solution to Φ \ C becomes a solution to Φ.

1. Prove that, throughout any such procedure, F is satisfiable ⇔ Φ is satisfiable ⇔ Φ ∪ Ψ is satisfiable.

2. Furthermore, given any solution to Φ, we obtain a solution to F by applying the erp rules in reverse order.

3. What is wrong with the following scenario? Start with one hard clause, (x), and no soft clauses. Reclassify (x) as soft, using the erp rule x ← 1. Then append a new soft clause (x¯).

4. If C is certifiable for Φ (see exercise 385), can we safely learn C?

5. If C is certifiable for Φ \ C, can we safely forget C?

6. In what cases is it legitimate to discard a clause, hard or soft, that is subsumed by another clause, hard or soft?

7. In what cases is self-subsumption permissible?

8. Explain how to eliminate all clauses that involve a particular variable x.

9. Show that, if z is a new variable, we can safely learn the three new soft clauses (x ∨ z), (y ∨ z), (x¯∨y¯∨z¯) in Tseytin’s concept of extended resolution.

384. [25] Continuing the previous exercise, show that we can always safely forget any clause C that contains a literal l for which C ◊ C′ is certifiable for Φ \ C whenever C′ ∊ Φ contains l¯. What erp rule is appropriate?

385. [22] Clause C is called certifiable for a set of clauses F if F∧C¯⊢1∈, as in (119). It is said to be absorbed by F if it is nonempty and F∧C⁢  \  l¯⊢1l for every l ∊ C, or if it is empty and F ⊢1 ∊. (Every clause of F is obviously absorbed by F.)

1. True or false: If C is absorbed by F, it is certifiable for F.

2. Which of {1¯,1¯2,1¯23} are implied by, certifiable for, or absorbed by R′ in (7)?

3. If C is certifiable for F and if all clauses of F are absorbed by F′, prove that C is certifiable for F′.

4. If C is absorbed by F and if all clauses of F are absorbed by F′, prove that C is absorbed by F′.

▸ 386. [M31] Let Algorithm C_0 be a variant of Algorithm C that (i) makes all decisions at random; (ii) never forgets a learned clause; and (iii) restarts whenever a new clause has been learned. (Thus, step C5 ignores Mp and Mf ; step C6 chooses l uniformly at random from among the 2(n−F ) currently unassigned literals; step C8 backjumps while F > i_1, instead of while F > id′+1; and after step C9 has stored a new clause, with d > 0, it simply sets d ← 0 and returns to C5. The data structures HEAP, HLOC, OVAL, ACT are no longer used.) We will prove that Algorithm C_0 is, nevertheless, quite powerful.

In the remainder of this exercise, F denotes the set of clauses known by Algorithm C_0, both original and learned; in particular, the unit clauses of F will be the first literals L_0, L_1, ... , Li1−1 on the trail. If C is any clause and if l ∊ C, we define
score⁢  (F,C,l)={∞|{l′|  F∧C⁢ \ l ¯⊢1  l′}|,⁢  if⁢   F∧C⁢   \   l¯⁢⊢1 l;otherwise.

Thus score(F, C, l) represents the total number of literals on the trail after making all the unforced decisions of C¯⁢ \ l¯, if no conflict arises. We say that Algorithm C_0 performs a “helpful round” for C and l if (i) every decision literal belongs to C¯; and (ii) l¯ is chosen as a decision literal only if the other elements of C¯ are already in the trail.

1. Let C be certifiable for F , and suppose that score(F, C, l) < ∞ for some l ∊ C. Prove that if F′ denotes F together with a clause learned on a round that’s helpful for C and l, then score(F′, C,l) > score(F, C, l).

2. Furthermore score(F′, C, l) ≥ score(F, C, l) after an unhelpful round.

3. Therefore C will be absorbed by the set F′ of known clauses after at most |C|n helpful rounds have occurred.

4. If |C| = k, show that Pr(helpful round) ≥ (k − 1)!/(2n)k ≥ 1/(4nk).

5. Consequently, by exercise 385(c), if there exists a certificate of unsatisfiability (C_1, ...,Ct) for a family of clauses F with n variables, Algorithm C_0 will prove F unsatisfiable after learning an average of μ≤4Σi=1t⁢|Ci|n1+|Ci| clauses. (And it will q.s. need to learn at most μn ln n clauses, by exercise MPR–102.)

▸ 387. [21] Graph G is said to be embedded in graph G′ if every vertex v of G corresponds to a distinct vertex v′ of G′, where u′ −−− v′ in G′ whenever u −−− v in G. Explain how to construct clauses that are satisfiable if and only if G can be embedded in G′.

388. [20] Show that the problems of deciding whether or not a given graph G (a) contains a k-clique, (b) can be k-colored, or (c) has a Hamiltonian cycle can all be regarded as graph embedding problems.

▸ 389. [22] In this 4 × 4 diagram, it’s possible to trace out the phrase ‘THE⊔ART⊔OF⊔COMPUTER⊔PROGRAMMING’ by making only king moves and knight moves, except for the final step from N to G.

Rearrange the letters so that the entire phrase can be traced.

▸ 390. [23] Let G be a graph with vertices V, edges E, |E| = m, |V | = n, and s, t ∊ V.

1. Construct O(kn) clauses that are satisfiable if and only if there’s a path of length k or less from s to t, given k.

2. Construct O(m) clauses that are satisfiable if and only if there’s at least one path from s to t.

3. Construct O(n^2) clauses that are satisfiable if and only if G is connected.

4. Construct O(km) clauses that are unsatisfiable if and only if there’s a path of length k or less from s to t, given k.

5. Construct O(m) clauses that are unsatisfiable if and only if there’s at least one path from s to t.

6. Construct O(m) clauses that are unsatisfiable if and only if G is connected. (This construction is much better than (c), in a sparse graph.)

391. [M25] The values of two integer variables satisfy 0 ≤ x, y < d, and they are to be represented as l-bit quantities xl−1 ...x_0, yl−1 ... y_0, where l = ⌈lg d⌉. Specify three different ways to encode the relation x ≠ y:

1. Let x = (xl−1 ... x_0)2 and y = (yl−1 ... y_0)2; and let the encoding enforce the conditions (xl−1 ...x_0)2 < d, and (yl−1 ... y_0)2 < d, as well as ensuring that x ≠ y by introducing 2l + 1 additional clauses in l auxiliary variables.

2. Like (a), but there are d additional clauses (not 2l + 1), and no auxiliaries.

3. All bit patterns xl−1 ...x_0 and yl−1 ...y_0 are valid, but some values might have two different patterns. The encoding has d clauses and no auxiliary variables.

392. [22] The blank spaces in the following diagrams can be filled with letters in such a way that all occurrences of the same letter are rookwise connected:

1. Demonstrate how to do it. (Puzzle (i) is easy; the others less so.)

2. Similarly, solve the following puzzles — but use kingwise connectedness instead.

3. Construct clauses with which a SAT solver can solve general puzzles of this kind: Given a graph G and disjoint sets of vertices T_1, T_2, ... , Tt, a solution should exhibit disjoint connected sets of vertices S_1, S_2, ... , St, with Tj ⊆ Sj for 1 ≤ j ≤ t.

393. [25] (T. R. Dawson, 1911.) Show that it’s possible for each white piece in the accompanying chess diagram to capture the corresponding black piece, via a path that doesn’t intersect any of the other paths. How can SAT help to solve this problem?

394. [25] One way to encode the at-most-one constraint S≤1(y_1, ...,yp) is to introduce l = ⌈lg p⌉ auxiliary variables together with the following nl + n − 2l clauses, which essentially “broadcast” the value of j when yj becomes true:

(y¯j∨(−1)bt  at)      for⁢  1≤j≤p, 1≤t≤q=⌊lg(2p−j)⌋, where 2p−j=(1b1⁢ ...⁢ bq)2.

For example, the clauses when p = 3 are (y¯1∨a1)∧(y¯1∨a¯2)∧(y¯2∨a1)∧(y¯2∨a2)∧(y¯3∨a¯1).

Experiment with this encoding by applying it to Langford’s problem, using it in place of (13) whenever p ≥ 7.

395. [20] What clauses should replace (15), (16), and (17) if we want to use the order encoding for a graph coloring problem?

▸ 396. [23] (Double clique hints.) If x has one of the d values {0, 1, ...,d − 1}, we can represent it binarywise with respect to two different orderings by letting xj = [x ≥ j] and x^j=[xπ≥j] for 1 ≤ j < d, where π is any given permutation. For example, if d = 4 and (0π, 1π, 2π, 3π) = (2, 3, 0, 1), the representations x1x2x3:x^1x^2x^3 of 0, 1, 2, and 3 are respectively 000:110, 100:111, 110:000, and 111:100. This double ordering allows us to encode graph coloring problems by including not only the hints (162) but also

(v^1d−k+1¯∨⋅⋅⋅∨v^kd−k+1¯⁢)∧(v^1k−1∨⋅⋅⋅∨v^kk−1),
whenever the vertices {v_1, ...,vk} form a k-clique.

Explain how to construct clauses for this encoding, and experiment with coloring the n × n queens graph when (0π, 1π, 2π, 3π, 4π,... ) = (0, d−1, 1, d−2, 2, ... ) is the inverse of the organ-pipe permutation.

▸ 397. [22] (N. Tamura, 2014.) Suppose x_0, x_1, ... , xp−1 are integer variables with the range 0 ≤ xi < d, represented in order encoding by Boolean variables xij=[xi≥j] for 0 ≤ i < p and 1 ≤ j < d. Show that the all-different constraint, “xi ≠ xj for 0 ≤ i < j < p,” can be nicely encoded by introducing auxiliary integer variables y_0, y_1, ... , yd−1 with the range 0 ≤ yj < p, represented in order encoding by Boolean variables yij=[yi≥j] for 1 ≤ i < p and 0 ≤ j < d, and by devising clauses to enforce the condition xi = j ⇒ yj = i. Furthermore, hints analogous to (162) can be given.

398. [18] Continuing exercise 397, what’s an appropriate way to enforce the alldifferent constraint when x_0, ... , xp−1 are represented in the direct encoding?

▸ 399. [23] If the variables u and v range over d values {1, ...,d}, it’s natural to encode them directly as sequences u_1 ...ud and v_1 ...vd, where ui = [u = i] and vj = [v = j], using the at-least-one clauses (15) and the at-most-one clauses (17). A binary constraint tells us which pairs (i, j) are legal; for example, the graph-coloring constraint says that i ≠ j when i and j are the colors of adjacent vertices in some graph.

One way to specify such a constraint is to assert the preclusion clauses (u¯i∨v¯j) for all illegal pairs (i, j), as we did for graph coloring in (16). But there’s also another general way: We can assert the support clauses

Λi=1d(u¯i∨V{vj|(i,j)⁢ is⁢ legal})∧Λj=1d(v¯j∨V{ui⁢| (i,j)  is⁢ legal})

instead. Graph coloring with d colors would then be represented by clauses such as (ū_3 ∨ v_1 ∨ v_2 ∨ v_4 ∨ ··· ∨ vd), when u and v are adjacent.

1. Suppose t of the d^2 pairs (i, j) are legal. How many preclusion clauses are needed? How many support clauses?

2. Prove that the support clauses are always at least as strong as the preclusion clauses, in the sense that all consequences of the preclusion clauses under unit propagation are also consequences of the support clauses under unit propagation, given any partial assignment to the binary variables {u_1, ...,ud, v_1, ...,vd}.

3. Conversely, in the case of the graph-coloring constraint, the preclusion clauses are also at least as strong as the support clauses (hence equally strong).

4. However, exhibit a binary constraint for which the support clauses are strictly stronger than the preclusion clauses.

400. [25] Experiment with preclusion clauses versus support clauses by applying them to the n queens problem. Use Algorithms L, C, and W for comparison.

401. [16] If x has the unary representation x^1x^2 ... xd−1, what is the unary representation of (a) y = ⌈x/2⌉? (b) z = ⌊(x +1)/3⌋?

402. [18] If x has the unary representation x^1x^2 ... xd−1, encode the further condition that x is (a) even; (b) odd.

403. [20] Suppose x, y, z have the order encoding, with 0 ≤ x, y, z < d. What clauses enforce (a) min(x, y) ≤ z? (b) max(x, y) ≤ z? (c) min(x, y) ≥ z? (d) max(x, y) ≥ z?

▸ 404. [21] Continuing exercise 403, encode the condition |x − y| ≥ a, for a given constant a ≥ 1, using d +1 − a clauses of length ≤ 4 and no auxiliary variables.

▸ 405. [M23] The purpose of this exercise is to encode the constraint ax + by ≤ c, when a, b, c are integer constants, assuming that x, y are order-encoded with range [0 ..d).

1. Prove that it suffices to consider cases where a, b, c > 0.

2. Exhibit a suitable encoding for the special case 13x − 8y ≤ 7, d = 8.

3. Exhibit a suitable encoding for the special case 13x − 8y ≥ 1, d = 8.

4. Specify an encoding that works for general a, b, c, d.

406. [M24] Order-encode (a) xy ≤ a and (b) xy ≥ a, when a is an integer constant.

▸ 407. [M22] If x, y, z are order-encoded, with 0 ≤ x, y < d and 0 ≤ z < 2d−1, the clauses

Λk=12d−2⁢   Λj=max(0,k+1−d)k(x¯j∨y¯k−j∨zk)

are satisfiable if and only if x+y ≤ z; this is the basic idea underlying (20). Another way to encode the same relation is to introduce new order-encoded variables u and v, and to construct clauses for the relations ⌊x/2⌋ + ⌊y/2⌋ ≤ u and ⌈x/2⌉ + ⌈y/2⌉ ≤ v, recursively using methods for numbers less than ⌈d/2⌉ and ⌊d/2⌋ + 1. Then we can finish the job by letting z^1 = v^1, z2d−2 = vd (d even) or ud−1 (d odd), and appending the clauses

(u¯j∨z2j)∧(v¯j+1∨z2j)∧(u¯j∨v¯j+1∨z2j+1),for⁢   1≤j≤d−2.

1. Explain why the alternative method is valid.

2. For what values of d does that method produce fewer clauses?

3. Consider analogous methods for the relation x + y ≥ z.

▸ 408. [25] (Open shop scheduling.) Consider a system of m machines and n jobs, together with an m×n matrix of nonnegative integer weights W = (wij) that represent the amount of uninterrupted time on machine i that is needed by job j.

The open shop scheduling problem seeks a way to get all the work done in t units of time, without assigning two jobs simultaneously to the same machine and without having two machines simultaneously assigned to the same job. We want to minimize t, which is called the “makespan” of the schedule.

For example, suppose m = n = 3 and W=(703172235). A “greedy” algorithm, which repeatedly fills the lexicographically smallest time slot (t, i, j) such that wij > 0 but neither machine i nor job j have yet been scheduled at time t, achieves a makespan of 12 with the following schedule:

1. Is 12 the optimum makespan for this W?

2. Prove that the greedy algorithm always produces a schedule whose makespan is less than (maxi=1mΣj=1nwij)+(maxj=1n⁢Σi=1m⁢ wij), unless W is entirely zero.

3. Suppose machine i begins to work on job j at time sij, when wij > 0. What conditions should these starting times satisfy, in order to achieve the makespan t?

4. Show that the order encoding of these variables sij yields SAT clauses that nicely represent any open shop scheduling problem.

5. Let ⌊W/k⌋ be the matrix obtained by replacing each element wij of W by ⌊wij/k⌋. Prove that if the open shop scheduling problem for ⌊W/k⌋ and t is unsatisfiable, so is the problem for W and kt.

▸ 409. [M26] Continuing exercise 408, find the best makespans in the following cases:

1. m = 3, n = 3r + 1; w1 j = w2(r+j) = w3(2r+j) = aj for 1 ≤ j ≤ r; w1n = w2n = w3n = ⌊(a_1 + ··· + ar)/2⌋; otherwise wij = 0. (The positive integers aj are given.)

2. m = 4, n = r +2; w1j = (r +1)aj and w2j = 1 for 1 ≤ j ≤ r; w2(n−1) = w2n = (r +1) ⌊(a_1 + ··· + ar)/2⌋; w3(n−1) = w4n = w2n + r; otherwise wij = 0.

3. m = n; wjj = n − 2, wjn = wnj = 1 for 1 ≤ j < n; otherwise wij = 0.

4. m = 2; w1j = aj and w2j = bj for 1 ≤ j ≤ n, where a_1 +··· +an = b_1 + ··· +bn = s and aj + bj ≤ s for 1 ≤ j ≤ n.

410. [24] Exhibit clauses for the constraint 13x−8y ≤ 7 when x and y are log-encoded as 3-bit integers x = (x_2x_1x_0)2 and y = (y_2y_1y_0)2. (Compare with exercise 405(b).)

▸ 411. [25] If x = (xm ...x_1)2, y = (yn ...y_1)2, and z = (zm+n ...z_1)2 stand for binary numbers, the text explains how to encode the relation xy = z with fewer than 20mn clauses, using Napier–Dadda multiplication. Explain how to encode the relations xy ≤ z and xy ≥ z with fewer than 9mn and 11mn clauses, respectively.

412. [40] Experiment with the encoding of somewhat large numbers by using a radix-d representation in which each digit has the order encoding.

413. [M22] Find all CNF formulas for the function (x_1 ⊕ y_1) ∨ ··· ∨ (xn ⊕ yn).

414. [M20] How many clauses will remain after the auxiliary variables a_1, ... , an−1 of (169) have been eliminated by resolution?

▸ 415. [M22] Generalize (169) to an encoding of lexicographic order on d-ary vectors, (x_1 ...xn)d ≤ (y_1 ...yn)d, where each xk=xk1⁢ +⋅⋅⋅+xkd−1 and yk=yk1⁢ +⋅⋅⋅+ykd−1 has the order encoding. What modifications to your construction will encode the strict relation x_1 ...xn < y_1 ...yn?

416. [20] Encode the condition ‘if x_1 ...xn = y_1 ...yn then u_1 ...um = v_1 ...vm’, using 2m+2n+1 clauses and n+1 auxiliary variables. Hint: 2n of the clauses are in (172).

417. [21] Continuing exercise 42, what is the Tseytin encoding of the ternary mux operation ‘s ← t? u: v’ ? Use it to justify the translation of branching programs via (173).

418. [23] Use a branching program to construct clauses that are satisfiable if and only if (xij) is an m × n Boolean matrix whose rows satisfy the hidden weighted bit function hn and whose columns satisfy the complementary function h¯m. In other words,
ri=Σj=1nxij,cj=Σi=1mxij⁢   and⁢  xiri=1,  xcjj=0, assu⁢ming⁢ that  xi0=x0j=0.

419. [M21] If m, n ≥ 3, find (by hand) all solutions to the problem of exercise 418 such that (a) ∑xij = m + 1 (the minimum); (b) ∑xij = mn − n − 1 (the maximum).

420. [18] Derive (175) mechanically (that is, “without thinking”) from the Boolean chain s ← x_1 ⊕ x_2, c ← x_1 ∧ x_2, t ← s ⊕ x_3, c′ ← s ∧ x_3, requiring c = c′ = 0.

421. [18] Derive (176) mechanically from the branching program I5=(1¯?4:3), I4=(2¯?1:2), I3=(2¯?2:0), I2=(3¯?1:0), beginning at I_5.

422. [11] What does unit propagation deduce when the additional clause (x_1) or (x_2) is appended to (a) F in (175)? (b) G in (176)?

423. [22] A representation F that satisfies a condition like (180) but with l replaced by can be called “weakly forcing.” Exercise 422 shows that (175) and (176) are weakly forcing. Does the BDD of every function define a weakly forcing encoding, via (173)?

▸ 424. [20] The dual of the Pi function has the prime clauses {1¯2¯3¯,1¯3¯4¯,23¯4¯,234,12} (see 7.1.1–(30)). Can any of them be omitted from a forcing representation?

425. [18] A clause with exactly one positive literal is called a definite Horn clause, and Algorithm 7.1.1C computes the “core” of such clauses. If F consists of definite Horn clauses, prove that x is in the core if and only if F ⊢1 x, if and only if F∧(x¯)⊢1∈.

▸ 426. [M20] Suppose F is a set of clauses that represent f(x_1, ...,xn) using auxiliary variables {a_1, ...,am} as in (170), where m > 0. Let G be the clauses that result after variable am has been eliminated as in (112).

1. True or false: If F is forcing then G is forcing.

2. True or false: If F is not forcing then G is not forcing.

427. [M30] Exhibit a function f(x_1, ...,xn) for which every set of forcing clauses that uses no auxiliary variables has size Ω(3n/n^2), although f can actually be represented by a polynomial number of forcing clauses when auxiliary variables are introduced. Hint: See exercise 7.1.1–116.

428. [M27] A generic graph G on vertices {1, ...,n} can be characterized by (n2) Boolean variables X = {xij | 1 ≤ i < j ≤ n}, where xij = [i−−−j in G]. Properties of G can therefore be regarded as Boolean functions, f(X).

1. Let fnd(X) = [χ(G) ≤ d]; that is, fnd is true if and only if G has a d-coloring. Construct clauses Fnd that represent the function fnd(X) ∨ y, using auxiliary variables Z = {zjk | 1 ≤ j ≤ n, 1 ≤ k ≤ d} that mean “vertex j has color k.”

2. Let Gnd be a forcing representation of the Boolean function Fnd(X, y, Z), and suppose that Gnd has M clauses in N variables. (These N variables should include the (n2)+1+nd variables of Fnd, along with an arbitrary number of additional auxiliaries.) Explain how to construct a monotone Boolean chain of cost O(MN^2) for the function f¯nd (see exercise 7.1.2–84), given the clauses of Gnd.

Note: Noga Alon and Ravi B. Boppana, Combinatorica 7 (1987), 1–22, proved that every monotone chain for this function has length exp Ω((n/ log n)1/3) when d +1 = ⌊(n/ lg n)2/3/4⌋. Hence M and N cannot both be of polynomial size.

429. [22] Prove that Bailleux and Boufkhad’s clauses (20), (21) are forcing: If any r of the x’s have been set to 1, then unit propagation will force all the others to 0.

430. [25] Similarly, Sinz’s clauses (18) and (19) are forcing.

▸ 431. [20] Construct efficient, forcing clauses for the relation x_1+ ··· +xm ≤ y_1+ ··· +yn.

432. [24] Exercise 404 gives clauses for the relation |x − y| ≥ a. Are they forcing?

▸ 433. [25] Are the lexicographic-constraint clauses in (169) forcing?

434. [21] Let Ll be the language defined by the regular expression 0∗1l0∗; in other words, the binary string x_1 ...xn is in Ll if and only if it consists of zero or more 0s followed by exactly l 1s followed by zero or more 0s.

1. Explain why the following clauses are satisfiable if and only if x_1 ...xn ∊ Ll:
(i) (p¯k∨x¯k), (p¯k∨pk−1), and (p¯k−1∨xk∨pk) for 1 ≤ k ≤ n, also (p_0); (ii) (q¯k∨x¯k), (q¯k∨qk+1), and (q¯k+1∨xk∨qk) for 1 ≤ k ≤ n, also (qn+1); (iii) (r¯k∨pk−1)∧Λ0≤d<l(r¯k∨xk+d)∧(r¯k∨qk+1) for 1 ≤ k ≤ n +1 − l, also (r_1 ∨ ··· ∨ rn+1−l).

2. Show that those clauses are forcing when l = 1 but not when l = 2.

▸ 435. [28] Given l ≥ 2, construct a set of O(n log l) clauses that characterize the language Ll of exercise 434 and are forcing.

436. [M32] (Nondeterministic finite-state automata.) A regular language L on the alphabet A can be defined in the following well-known way: Let Q be a finite set of “states,” and let I ⊆ Q and O ⊆ Q be designated “input states” and “output states.” Also let T ⊆ Q×A×Q be a set of “transition rules.” Then the string x_1 ...xn is in L if and only if there’s a sequence of states q_0, q_1, ... , qn such that q_0 ∊ I, (qk−1, xk, qk) ∊ T for 1 ≤ k ≤ n, and qn ∊ O.

Given such a definition, where A = {0, 1}, use auxiliary variables to construct clauses that are satisfiable if and only if x_1 ...xn ∊ L. The clauses should be forcing, and there should be at most O(n|T |) of them.

As an example, write out the clauses for the language L_2 = 0∗1^20∗ of exercise 434.

437. [M21] Extend exercise 436 to the general case where A has more than two letters.

438. [21] Construct a set of forcing clauses that are satisfiable if and only if a given binary string x_1 ...xn contains exactly t runs of 1s, having lengths (l_1, l_2, ...,lt) from left to right. (Equivalently, the string x_1 ...xn should belong to the language defined by the regular expression 0∗1l_1 0+ 1l_2 0+ ... 0+1lt 0∗.)

▸ 439. [30] Find efficient forcing clauses for the constraint that x_1 + ··· + xn = t and that there are no two consecutive 1s. (This is the special case l_1 = ··· = lt = 1 of the previous exercise, but a much simpler construction is possible.)

440. [M33] Extend exercise 436 to context free languages, which can be defined by a set S ⊆ N and by production rules U and W of the following well-known forms: U ⊆ {P → a | P ∊ N, a ∊ A} and W ⊆ {P → QR | P, Q, R ∊ N}, where N is a set of “nonterminal symbols.” A string x_1 ...xn with each xj ∊ A belongs to the language if and only if it can be produced from a nonterminal symbol P ∊ S.

441. [M35] Show that any threshold function f(x_1, ...,xn) = [w_1x_1 + ··· + wnxn ≥ t] has a forcing representation whose size is polynomial in log |w_1| + ··· + log |wn|.

▸ 442. [M27] The unit propagation relation ⊢1 can be generalized to kth order propagation ⊢k as follows: Let F be a family of clauses and let l be a literal. If (l_1, l_2, ...,lp) is a sequence of literals, we write Lq−={l1,...,lq−1,l¯q} for 1 ≤ q ≤ p. Then

F⊢0l⇔∈∊F;F⊢k+1 l⇔F|L1−⊢k∈,F⁢| L2−⊢k  ∈, ...⁢,and⁢ F| Lp−⊢k∈ ⁢           for some strictly distinct literals l1,l2,...⁢ ,lp⁢ with lp=l;F⊢k ∈⇔F⊢k l⁢   and F⊢k   l¯⁢   for some literal l.

1. Verify that ⊢1 corresponds to unit propagation according to this definition.

2. Describe ⊢2 informally, using the concept of “failed literals.”

3. Prove that F ⊢k or F⊢k  l¯ implies F | l ⊢k ∊ for all literals l, and furthermore that F ⊢k ∊ implies F ⊢k+1 ∊, for all k ≥ 0.

4. True or false: F ⊢k l implies F ⊢k+1 l.

5. Let Lk(F ) = {l | F ⊢k l}. What is Lk(R′), where R′ appears in (7) and k ≥ 0? f) Given k ≥ 1, explain how to compute Lk(F ) and F | Lk(F ) in O(n2k−1m) steps, when F has m clauses in n variables.

443. [M24] (A hierarchy of hardness.) Continuing the previous exercise, a family of clauses F is said to belong to class UCk if it has the property that

F | L ⊢ ∊ implies F | L ⊢k ∊ for all sets of strictly distinct literals L.

(“Whenever a partial assignment yields unsatisfiable clauses, the inconsistency can be detected by kth order propagation.”) And F is said to belong to class PCk if

F | L ⊢ l implies F | L ⊢k l for all sets of strictly distinct literals L ∪ l.

1. Prove that PC_0 ⊂ UC_0 ⊂ PC_1 ⊂ UC_1 ⊂ PC_2 ⊂ UC_2 ⊂ ··· , where the set inclusions are strict (each class is contained in but unequal to its successor).

2. Describe all families F that belong to the smallest class, PC_0.

3. Give interesting examples of families in the next smallest class, UC_0.

4. True or false: If F contains n variables, F ∊ PCn.

5. True or false: If F contains n variables, F ∊ UCn−1.

6. Where do the clauses R′ of (7) fall in the hierarchy?

444. [M26] The following single lookahead unit resolution algorithm, called SLUR, returns either ‘sat’, ‘unsat’, or ‘maybe’, depending on whether a given set F of clauses is satisfiable, unsatisfiable, or beyond its ability to decide via easy propagations:

E1. [Propagate.] If F ⊢1 , terminate (‘unsat’). Otherwise set F ← F |{l | F ⊢1 l}.

E2. [Satisfied?] If F = ∅, terminate (‘sat’). Otherwise set l to any literal within F.

E3. [Lookahead and propagate.] If F|  l⁢  ⊬1∈, set F ← F | l |{l′ | F | l ⊢1 l′} and return to E2. Otherwise if F|  l¯⁢  ⊬1∈, F←F|l¯|{l′ |F| l¯⁢ ⊢1  l′⁢} and return to E2. Otherwise terminate (‘maybe’).

Notice that this algorithm doesn’t backtrack after committing itself in E2 to either l or l¯.

1. If F consists of Horn clauses, possibly renamed (see exercise 7.1.1–55), prove that SLUR will never return ‘maybe’, regardless of how it chooses l in step E2.

2. Find four clauses F on three variables such that SLUR always returns ‘sat’, although F is not a set of possibly renamed Horn clauses.

3. Prove that SLUR never returns ‘maybe’ if and only if F ∊ UC_1 (see exercise 443).

4. Explain how to implement SLUR in linear time with respect to total clause length.

▸ 445. [22] Find short certificates of unsatisfiability for the pigeonhole clauses (106)–(107), when they are supplemented by (a) (181); (b) (182); (c) (183).

446. [M10] What’s the maximum number of edges in a subgraph of Km,n that has girth ≥ 6? (Express your answer in terms of Z(m, n).)

▸ 447. [22] Determine the maximum number of edges in a girth-8 subgraph of K8,8.

448. [M25] What is Z(m, n) when m is odd and n = m(m−1)/6? Hint: See 6.5–(16).

449. [21] Exhibit n × n quad-free matrices that contain the maximum number of 1s and obey the lexicographic constraints (185), (186), for 8 ≤ n ≤ 16.

450. [25] Prove that there is essentially only one 10 × 10 quad-free system of points and lines with 34 incidences. Hint: First show that every line must contain either 3 points or 4 points; hence every point must belong to either 3 lines or 4 lines.

▸ 451. [28] Find a way to color the squares of a 10 × 10 board with three colors, so that no rectangle has four corners of the same color. Prove furthermore that every such “nonchromatic rectangle” board has the color distribution {34, 34, 32}, not {34, 33, 33}. But show that if any square of the board is removed, a nonchromatic rectangle is possible with 33 squares of each color.

452. [34] Find a nonchromatic rectangle with four colors on an 18 × 18 board.

453. [M23] An m×n matrix X = (xij) is said to be decomposable if it has row indices R ⊆ {1, ...,m} and column indices C ⊆ {1, ...,n} such that 0 < |R| + |C| < m + n, with xij = 0 whenever (i ∊ R and j ∉ C) or (i ∉ R and j ∊ C). It represents a bipartite graph on the vertices {u_1, ...,um} and {v_1, ...,vn}, if [ui−−−vj] = [xij ≠ 0].

1. Prove that X is indecomposable if and only if its bipartite graph is connected.

2. The direct sum X′ ⊕ X′ of matrices X′ and X″, where X′ is m′ × n′ and X″ is m″ × n″, is the (m′ + m″) × (n′ + n″) “block diagonal” matrix X that has X′ in its upper left corner, X″ in the lower right corner, and zeros elsewhere (see 7–(40)). True or false: If the rows and columns of X′ and X″ are nonnegative and lexicographically ordered as in (185) and (186), so are the rows and columns of X.

3. Let X be any nonnegative matrix whose rows and columns are lexicographically nonincreasing, as in (185) and (186). True or false: X is decomposable if and only if X is a direct sum of smaller matrices X′ and X″.

454. [15] If τ is an endomorphism for the solutions of f, show that f(x) = f(xτ ) for every cyclic element x (every element that’s in a cycle of τ ).

455. [M20] Suppose we know that (187) is an endomorphism of some given clauses F on the variables {x_1, x_2, x_3, x_4}. Can we be sure that F is satisfiable if and only if F ∧ C is satisfiable, when (a) C=1¯24¯, i.e., C=(x¯1∨x2∨x¯4)? (b) C=23¯4¯? (c) C = 123? (d) C=13¯4?

456. [M21] For how many functions f(x_1, x_2, x_3, x_4) is (187) an endomorphism?

457. [HM19] Show that every Boolean f(x_1, x_2, x_3, x_4) has more than 51 quadrillion endomorphisms, and an n-variable function has more than 22n(n−1).

458. [20] The simplification of clauses by removing an autarky can be regarded as the exploitation of an endomorphism. Explain why.

▸ 459. [20] Let Xij denote the submatrix of X consisting of the first i rows and the first j columns. Show that the numbers sweep(Xij) satisfy a simple recurrence, from which it’s easy to compute sweep(X) = sweep(Xmn).

460. [21] Given m, n, k, and r, construct clauses that are satisfied by an m×n binary matrix X = (xij) if and only if sweep(X) ≤ k and ∑i,j xij ≥ r.

461. [20] What additional clauses will rule out non-fixed points of τ_1 and τ_2?

462. [M22] Explain why τ_1, τ_2, and τ_3 preserve satisfiability in the sweep problem.

▸ 463. [M21] Show that X is a fixed point of τ_1, τ_2, and τ_3 if and only if its rows and columns are nondecreasing. Therefore the maximum of νX = ∑i,j xij over all binary matrices of sweep k is a simple function of m, n, and k.

▸ 464. [M25] Transformations τ_1 and τ_2 don’t change the text’s example 10×10 matrix. Prove that they will never change any 10 × 10 matrix of sweep 3 that has ν X = 51.

465. [M21] Justify the text’s rule for simultaneous endomorphisms in the perfect matching problem: Any perfect matching must lead to one that’s fixed by every τuv.

466. [M23] Prove that when mn is even, the text’s even-odd rule (190) for endomorphisms of m × n domino coverings has exactly one fixed point.

467. [20] Mutilate the 7×8 and 8×7 boards by removing the upper right and lower left cells. What domino coverings are fixed by all the even-odd endomorphisms like (190)?

468. [20] Experiment with the mutilated chessboard problem when the even-odd endomorphisms are modified so that (a) they use the same rule for all i and j; or (b) they each make an independent random choice between horizontal and vertical.

▸ 469. [M25] Find a certificate of unsatisfiability (C_1, C_2, ...,Ct) for the fact that an 8 × 8 chessboard minus cells (1, 8) and (8, 1) cannot be exactly covered by dominoes hij and vij that are fixed under all of the even-odd endomorphisms. Each Ck for 1 ≤ k < t should be a single positive literal. (Therefore the clauses for this problem belong to the relatively simple class PC_2 in the hierarchy of exercise 443.)

▸ 470. [M22] Another class of endomorphisms, one for every 4-cycle, can also be used in perfect matching problems: Let the vertices (instead of the edges) be totally ordered in some fashion. Every 4-cycle can be written v_0 −−− v_1 −−− v_2 −−− v_3 −−− v_0, with v_0 > v_1 > v_3 and v_0 > v_2; the corresponding endomorphism changes any solution for which v_0v_1 = v_2v_3 = 1 by setting v_0v_1 ← v_2v_3 ← 0 and v_1v_2 ← v_3v_0 ← 1. Prove that every perfect matching leads to a fixed point of all these transformations.

471. [16] Find all fixed points of the mappings in exercise 470 when the graph is K2n.

472. [M25] Prove that even-odd endomorphisms such as (190) in the domino covering problem can be regarded as instances of the endomorphisms in exercise 470.

▸ 473. [M23] Generalize exercise 470 to endomorphisms for the unsatisfiable clauses of Tseytin’s graph parity problems in exercise 245.

474. [M20] A signed permutation σ is a symmetry of f(x) if and only if f(x) = f(xσ) for all x, and it is an antisymmetry if and only if we have f(x)=f¯(xσ) for all x.

1. How many signed permutations of n elements are possible?

2. Write 751¯4¯2¯63¯ in cycle form, as an unsigned permutation of {1,...,7, 1¯,...,7¯}.

3. For how many functions f of four variables is 4¯132¯ a symmetry?

4. For how many functions f of four variables is 4¯132¯ an antisymmetry?

5. For how many f(x_1, ...,x_7) is 751¯4¯2¯63¯ a symmetry or antisymmetry?

475. [M22] Continuing exercise 474, a Boolean function is called asymmetric if the identity is its only symmetry; it is totally asymmetric if it is asymmetric and has no antisymmetries.

1. If f is totally asymmetric, how many functions are equivalent to f under the operations of permuting variables, complementing variables, and/or complementing the function?

2. According to (a) and 7.1.1–(95), the function (x ∨ y) ∧ (x ⊕ z) is not totally asymmetric. What is its nontrivial symmetry?

3. Prove that if f is not asymmetric, it has an automorphism of prime order p.

4. Show that if (uvw)(u¯v¯w¯) is a symmetry of f, so is (uv)(u¯v¯).

5. Make a similar statement if f has a symmetry of the form (uvwxy)(u¯v¯w¯x¯y¯).

6. Conclude that, if n ≤ 5, the Boolean function f(x_1, ...,xn) is totally asymmetric if and only if no signed involution is a symmetry or antisymmetry of f.

7. However, exhibit a counterexample to that statement when n = 6.

476. [M23] For n ≤ 5, find Boolean functions of n variables that are (a) asymmetric but not totally asymmetric; (b) totally asymmetric. Furthermore, your functions should be the easiest to evaluate (in the sense of having a smallest possible Boolean chain), among all functions that qualify. Hint: Combine exercises 475 and 477.

▸ 477. [23] (Optimum Boolean evaluation.) Construct clauses that are satisfiable if and only if there is an r-step normal Boolean chain that computes m given functions g_1, ... , gm on n variables. (For example, if n = 3 and g_1 = 〈x_1x_2x_3〉, g_2 = x_1 ⊕ x_2 ⊕ x_3, such clauses with r = 4 and 5 enable a SAT solver to discover a “full adder” of minimum cost; see 7.1.2–(1) and 7.1.2–(22).) Hint: Represent each bit of the truth tables.

▸ 478. [23] Suggest ways to break symmetry in the clauses of exercise 477.

▸ 479. [25] Use SAT technology to find optimum circuits for the following problems:

1. Compute z_2, z_1, and z_0, when x_1 + x_2 + x_3 + x_4 = (z_2z_1z_0)2 (see 7.1.2–(27)).

2. Compute z_2, z_1, and z_0, when x_1 + x_2 + x_3 + x_4 + x_5 = (z_2z_1z_0)2.

3. Compute all four symmetric functions S_0, S_1, S_2, S_3 of {x_1, x_2, x_3}.

4. Compute all five symmetric functions S_0, S_1, S_2, S_3, S_4 of {x_1, x_2, x_3, x_4}.

5. Compute the symmetric function S_3(x_1, x_2, x_3, x_4, x_5, x_6).

6. Compute the symmetric function S0,4(x_1, ...,x_6) = [(x_1 + ··· + x_6) mod 4 = 0].

7. Compute all eight minterms of {x_1, x_2, x_3} (see 7.1.2–(30)).

480. [25] Suppose the values 0, 1, 2 are encoded by the two-bit codes xlxr = 00, 01, and 1∗, respectively, where 10 and 11 both represent 2. (See Eq. 7.1.3–(120).)

1. Find an optimum circuit for mod 3 addition: zlzr = (xlxr + ylyr)mod 3.

2. Find an optimum circuit that computes zlzr = (x_1 + x_2 + x_3 + ylyr)mod 3.

3. Conclude that [x_1 + ··· + xn ≡ a (modulo 3)] can be computed in < 3n steps.

▸ 481. [28] An ordered bit pair xy can be encoded by another ordered bit pair ⟦xy⟧ = (x⊕y) y without loss of information, because ⟦xy⟧ = uv implies ⟦uv⟧ = xy.

1. Find an optimum circuit that computes (⟦zz′⟧)2 = x_1 + x_2 + x_3.

2. Let ν⟦uv⟧ = (u ⊕ v)+ v, and note that ν⟦00⟧ = 0, ν⟦01⟧ = 2, ν⟦1∗⟧ = 1. Find an optimum circuit that, given x_1 ...x_5, computes z_1z_2z_3 such that we have ν⟦x_1x_2⟧ + ν⟦x_3x_4⟧ + x_5 = 2ν⟦z_1z_2⟧ + z_3.

3. Use that circuit to prove by induction that the “sideways sum” (z⌊lg n⌋ ...z_1 z_0)2 = x_1 + x_2 + ··· + xn can always be computed with fewer than 4.5n gates.

▸ 482. [26] (Erd˝os discrepancy patterns.) The binary sequence y_1 ...yt is called strongly balanced if we have |Σj=1k(2yi−1)|≤2 for 1 ≤ k ≤ t.

1. Show that this balance condition needs to be checked only for odd k ≥ 3.

2. Describe clauses that efficiently characterize a strongly balanced sequence.

3. Construct clauses that are satisfied by x_1x_2 ...xn if and only if xdx2d ...x⌊n/d⌋d is strongly balanced for 1 ≤ d ≤ n.

483. [21] Symmetry between colors was broken in the coloring problems of Table 6 by assigning fixed colors to a large clique in each graph. But many graphs have no large clique, so a different strategy is necessary. Explain how to encode the “restricted growth string” principle (see Section 7.2.1.5) with appropriate clauses, given an ordering v_1v_2 ...vn of the vertices: The color of vj must be at most one greater than the largest color assigned to {v_1, ...,vj−1}. (In particular, v_1 always has color 1.)

Experiment with this scheme by applying it to the Mycielski graphs of exercise 7.2.2.1–116.

484. [22] (Graph quenching.) A graph with vertices (v_1, ...,vn) is called “quenchable” if either (i) n = 1; or (ii) there’s a k such that vk −−− vk+1 and the graph on (v_1, ...,vk−1, vk+1, ...,vn) can be quenched; or (iii) there’s an l such that vl −−− vl+3 and the graph on (v_1, ...,vl−1, vl+3, vl+1, vl+2, vl+4, ...,vn) can be quenched.

1. Find a 4-element graph that is quenchable although v3―v4.

2. Construct clauses that are satisfiable if and only if a given graph is quenchable. Hint: Use the following three kinds of variables for this model-checking problem: xt,i,j = [vi−−−vj at time t], for 1 ≤ i < j ≤ n−t; qt,k = [a quenching move of type (ii) leads to time t + 1]; st,l = [a quenching move of type (iii) leads to time t + 1].

▸ 485. [23] Sometimes successive transitions in the previous exercise are commutative: For example, the effect of qt,k and qt+1,k+1 is the same as qt,k+2 and qt+1,k. Explain how to break symmetry in such cases, by allowing only one of the two possibilities.

486. [21] (Late Binding Solitaire.) Shuffle a deck and deal out 18 cards; then try to reduce these 18 piles to a single pile, using a sequence of “captures” in which one pile is placed on top of another pile. A pile can capture only the pile to its immediate left, or the pile found by skipping left over two other piles. Furthermore a capture is permitted only if the top card in the capturing pile has the same suit or the same rank as the top card in the captured pile. For example, consider the following deal:

Click here to view code image

J ♡ 5 ♡ 10♣ 8 ♢ J ♣ A ♣ K ♠ A ♡ 4♣ 8♠ 5♠ 5 ♢ 2 ♢ 10♠ A♠ 6♡ 3♡ 10♢

Ten captures are initially possible, including 5♡ × J♡, A♣××10♣, and 5♢ × 5♠. Some captures then make others possible, as in 8♠ ×× K♠×× 8♢.

If captures must be made “greedily” from left to right as soon as possible, this game is the same as the first 18 steps of a classic one-player game called “Idle Year,” and we wind up with five piles [see Dick’s Games of Patience (1883), 50–52]. But if we cleverly hold back until all 18 cards have been dealt, we can do much better.

Show that one can win from this position, but not if the first move is A♣× J♣.

▸ 487. [27] There are (648)=4426165368 ways to place eight queens on a chessboard. Long ago, W. H. Turton asked which of them causes the maximum number of vacant squares to remain unattacked. [See W. W. Rouse Ball, Mathematical Recreations and Problems, third edition (London: Macmillan, 1896), 109–110.]

Every subset S of the vertices of a graph has three boundary sets defined thus:

∂S=the set of all edges with exactly one endpoint ∈S;∂out S=the set of all vertices ∉S⁢  with at least one neighbor ∈S;∂in S=the set of all vertices ∈S⁢ with at least one neighbour ∉S.

Find the minimum and maximum sizes of ∂S, ∂outS, and ∂inS, over all 8-element sets S in the queen graph Q_8 (exercise 7.1.4–241). Which set answers Turton’s question?

▸ 488. [24] (Peaceable armies of queens.) Prove that armies of nine white queens and nine black queens can coexist on a chessboard without attacking each other, but armies of size 10 cannot, by devising appropriate sets of clauses and applying Algorithm C. Also examine the effects of symmetry breaking. (This problem has sixteen symmetries, because we can swap colors and/or rotate and/or reflect the board.) How large can coexisting armies of queens be on n × n boards, for n ≤ 11?

489. [M21] Find a recurrence for Tn, the number of signed involutions on n elements.

▸ 490. [15] Does Theorem E hold also when p_1p_2 ...pn is any signed permutation?

▸ 491. [22] The unsatisfiable clauses R in (6) have the signed permutation 2341¯ as an automorphism. How can this fact help us to verify their unsatisfiability?

492. [M20] Let τ be a signed mapping of the variables {x_1, ...,xn}; for example, the signed mapping ′4¯133¯′ stands for the operation (x1,x2,x3,x4)↦(x4¯,x1,x3,x3¯)=(x¯4,x1,x3,x¯3). When a signed mapping is applied to a clause, some of the resulting literals might coincide; or two literals might become complementary, making a tautology. When τ=4¯133¯, for instance, we have (123)τ=4¯13, (134¯)τ=4¯3, (134¯)τ=℘.

A family F of clauses is said to be “closed” under a signed mapping τ if Cτ is subsumed by some clause of F whenever C ∊ F . Prove that τ is an endomorphism of F in such a case.

493. [20] The problem waerden (3, 3; 9) has four symmetries, because we can reflect and/or complement all the variables. How can we speed up the proof of unsatisfiability by adding clauses to break those symmetries?

494. [21] Show that if (uvw)(u¯v¯w¯) is a symmetry of some clauses F, we’re allowed to break symmetries as if (uv)(u¯v¯), (uw)(u¯w¯), and (vw)(v¯w¯) were also symmetries. For example, if i < j < k and if (ijk)(i¯j¯k¯) is a symmetry, we can assert (x¯i∨xj)∧(x¯j∨xk) with respect to the global ordering p_1 ...pn = 1 ...n. What are the corresponding binary clauses when the symmetry is (i) (ijk¯)(i¯j¯k)? (ii) (ij¯k)(i¯jk¯)? (iii) (ij¯k¯)(i¯j¯k)?

495. [M22] Spell out the details of how we can justify appending clauses to assert (185) and (186), using Corollary E, whenever we have an m × n problem whose variables xij possess both row and column symmetry. (In other words we assume that xij ↦ x(iπ)(jρ) is an automorphism for all permutations π of {1, ...,m} and ρ of {1, ...,n}.)

▸ 496. [M20] B. C. Dull reasoned as follows: “The pigeonhole clauses have row and column symmetry. Therefore we can assume that the rows are lexicographically increasing from top to bottom, and the columns are lexicographically increasing from right to left. Consequently the problem is easily seen to be unsatisfiable.” Was he correct?

497. [22] Use BDD methods to determine the number of 8 × 8 binary matrices that have both rows and columns in nondecreasing lexicographic order. How many of them have exactly r 1s, for r = 24, r = 25, r = 64 − 25 = 39, and r = 64 − 24 = 40?

498. [22] Justify adding the symmetry-breakers (183) to the pigeonhole clauses.

499. [21] In the pigeonhole problem, is it legitimate to include the clauses (183) together with clauses that enforce lexicographic row and column order?

500. [16] The precocious student J. H. Quick decided to extend the monkey wrench principle, arguing that if F_0 ∪ S ⊢ l then the original clauses F can be replaced by F | l. But he soon realized his mistake. What was it?

501. [22] Martin Gardner introduced an interesting queen placement problem in Scientific American 235, 4 (October 1976), 134–137: “Place r queens on an m × n chess-board so that (i) no three are in the same row, column, or diagonal; (ii) no empty square can be occupied without breaking rule (i); and (iii) r is as small as possible.” Construct clauses that are satisfiable if and only if there’s a solution to conditions (i) and (ii) with at most r queens. (A similar problem was considered in exercise 7.1.4–242.)

502. [16] (Closest strings.) Given binary strings s_1, ... , sm of length n, and threshold parameters r_1, ... , rm, construct clauses that are satisfiable by x = x_1 ...xn if and only if x differs from sj in at most rj positions, for 1 ≤ j ≤ m.

503. [M20] (Covering strings.) Given sj and rj as in exercise 502, show that every string of length n is within rj bits of some sj if and only if the closest string problem has no solution with parameters rj′=n−1−rj.

▸ 504. [M21] The problem in exercise 502 can be proved NP-complete as follows:

1. Let wj be the string of length 2n that is entirely 0 except for 1s in positions 2j − 1 and 2j, and let wn+j=w¯j, for 1 ≤ j ≤ n. Describe all binary strings of length 2n that differ from each of w_1, ... , w2n in at most n bit positions.

2. Given a clause (l_1 ∨ l_2 ∨ l_3) with strictly distinct literals l1,l2,l3∈{x1,...,xn, x¯1,...,x¯n}, let y be the string of length 2n that is entirely zero except that it has 1 in position 2k − 1 when some li is x¯k, and 1 in position 2k when some li is xk. In how many bit positions does a string that satisfies (a) differ from y?

3. Given a 3SAT problem F with m clauses and n variables, use (a) and (b) to construct strings s_1, ... , sm+2n of length 2n such that F is satisfiable if and only if the closest string problem is satisfiable with rj = n +[j> 2n].

4. Illustrate your construction in (c) by exhibiting the closest string problems that correspond to the simple 3SAT problems R and R′ in (6) and (7).

505. [21] Experiment with making Algorithm L nondeterministic, by randomizing the initial order of VAR in step L1 just as HEAP is initialized randomly in step C1. How does the modified algorithm perform on, say, problems D3, K0, and W2 of Table 6?

506. [22] The weighted variable interaction graph of a family of clauses has one vertex for each variable and the weight ∑2/(|c|(|c|− 1)) between vertices u and v, where the sum is over all clauses c that contain both ±u and ±v. Figure 95 indicates these weights indirectly, by making the heavier edges darker.

1. True or false: The sum of all edge weights is the total number of clauses.

2. Explain why the graph for test case B2 has exactly 6 edges of weight 2. What are the weights of the other edges in that graph?

▸ 507. [21] (Marijn Heule.) Explain why “windfalls” (see (72)) help Algorithm L to deal with miter problems such as D5.

508. [M20] According to Table 7, Algorithm C proved problem T3 to be unsatisfiable after learning about 323 thousand clauses. About how many times did it enter a purging phase in step C7?

509. [20] Several of the “training set” tasks used when tuning Algorithm C’s parameters were taken from the 100 test cases of Table 6. Why didn’t this lead to a problem of “overfitting” (namely, of choosing parameters that are too closely associated with the trainees)?

510. [18] When the data points A1, A2, ... , X8 were plotted in Fig. 98, one by one, they sometimes covered parts of previously plotted points, because of overlaps. What test cases are partially hidden by (a) T2? (b) X6? (c) X7?

511. [22] Problem P4 in Table 6 is a strange set of clauses that lead to extreme behavior of Algorithm C in Figs. 97 and 98; and it causes Algorithm L to “time out” in Fig. 96.

1. The preprocessing algorithm of the text needs about 1.5 megamems to convert those 2509 clauses in 400 variables into just 2414 clauses in 339 variables. Show empirically that Algorithm L makes short work of the resulting 2414 clauses.

2. How efficient is Algorithm C on those preprocessed clauses?

3. What is the behavior of WalkSAT on P4, with and without preprocessing?

512. [29] Find parameters for Algorithm C that will find an Erd˝os discrepancy pattern x_1x_2 ...xn rapidly when n = 500. (This is problem E0 in Table 6.) Then compare the running times of nine random runs with your parameters versus nine random runs with (194), when n = 400, 500, 600, ... , 1100, 1160, and 1161.

513. [24] Find parameters for Algorithm L that tune it for rand (3, m,n, seed).

514. [24] The timings quoted in the text for Algorithm W, for problems in Table 6, are based on the median of nine runs using the parameters p = .4 and N = 50n, restarting from scratch if necessary until a solution is found. Those parameters worked fine in most cases, unless Algorithm W was unsuited to the task. But problem C9 was solved more quickly with p = .6 and N = 2500n (943 Mμ versus 9.1Gμ).

Find values of p and N/n that give near-optimum performance for problem C9.

▸ 515. [23] (Hard sudoku.) Specify SAT clauses with which a designer of sudoku puzzles can meet the following specifications: (i) If cell (i, j) of the puzzle is blank, so is cell (10−i, 10−j), for 1 ≤ i, j ≤ 9. (ii) Every row, every column, and every box contains at least one blank. (Here “box” means one of sudoku’s nine special 3 × 3 subarrays.) (iii) No box contains an all-blank row or an all-blank column. (iv) There are at least two ways to fill every blank cell, without conflicting with nonblank entries in the same row, column, or box. (v) If a row, column, or box doesn’t already contain k, there are at least two places to put k into that row, column, or box, without conflict. (vi) If the solution has a 2 × 2 subarray of the form kllk, those four cells must not all be blank.

(Condition (i) is a feature of “classic” sudoku puzzles. Conditions (iv) and (v) ensure that the corresponding exact cover problem has no forced moves; see Section 7.2.2.1. Condition (vi) rules out common cases with non-unique solutions.)

516. [M49] Prove or disprove the strong exponential time hypothesis: “If τ < 2, there is an integer k such that no randomized algorithm can solve every kSAT problem in fewer than τn steps, where n is the number of variables.”

517. [25] Given clauses C_1, ... , Cm, the one-per-clause satisfiability problem asks if there is a Boolean assignment x_1 ...xn such that every clause is satisfied by a unique literal. In other words, we want to solve the simultaneous equations ∑Cj = 1 for 1 ≤ j ≤ m, where ∑C is the sum of the literals of clause C.

1. Prove that this problem is NP-complete, by reducing 3SAT to it.

2. Prove that this problem, in turn, can be reduced to its special case “one-in-three satisfiability,” where every given clause is required to be ternary.

518. [M32] Given a 3SAT problem with m clauses and n variables, we shall construct a (6m + n) × (6m + n) matrix M of integers such that the permanent, per M, is zero if and only if the clauses are unsatisfiable. For example, the solvable problem (7) corresponds to the 46 × 46 matrix indicated here; each shaded box stands for a fixed 6 × 6 matrix A that corresponds to a clause.

Each A has three “inputs” in columns 1, 3, 5 and three “outputs” in rows 2, 4, 6. The first n rows and the last n columns correspond to variables. Outside of the As, all entries are either 0 or 2; and the 2s link variables to clauses, according to a scheme much like the data structures in several of the algorithms in this section: Let Iij and Oij denote the jth input and output of clause i, for 1 ≤ i ≤ m and 1 ≤ j ≤ 3. Then, if literal l appears in t ≥ 0 clauses i_1 < ··· < it, as element j_1, ... , jt, we put ‘2’ in column Iik+1jk+1 of row Oikjk for 0 ≤ k ≤ t (Oi_0j is row |l|, Iit+1j is column 6m+|l|).

1. Find a 6 × 6 matrix A = (aij), whose elements are either 0, 1, or −1, such that
per (a11a12a13a14a15a16a21+2pa22a23+2qa24a25+2ra26a31a32a33a34a35a36a41+2ua42a43+2va44a45+2wa46a51a52a53a54a55a56a61+2xa62a63+2ya64a65+2za66)=16(per(p+1qruv+1wxyz+1)−1).

Hint: There’s a solution with lots of symmetry.

2. In which of the rows and columns of M does ‘2’ occur twice? once? not at all?

3. Conclude that per M = 24m+ns, when the 3SAT problem has exactly s solutions.

519. [20] Table 7 shows inconclusive results in a race for factoring between factor fifo and factor lifo . What is the comparable performance of factor_rand (m, n, z, 314159)?

▸ 520. [24] Every instance of SAT corresponds in a natural way to an integer programming feasibility problem: To find, if possible, integers x_1, ... , xn that satisfy the linear inequalities 0 ≤ xj ≤ 1 for 1 ≤ j ≤ n and
l1+l2+⋅⋅⋅+lk≥1for each clause     C=(l1∨l2∨⋅⋅⋅∨lk).

For example, the inequality that corresponds to the clause (x1∨x¯3∨x¯4∨x7) is x_1 +(1−x_3)+(1−x_4)+ x_7 ≥ 1; i.e., x_1 − x_3 − x_4 + x_7 ≥ −1.

Sophisticated “IP solvers” have been developed by numerous researchers for solving general systems of integer linear inequalities, based on techniques of “cutting planes” in high-dimensional geometry. Thus we can solve any satisfiability problem by using such general-purpose software, as an alternative to trying a SAT solver.

Study the performance of the best available IP solvers, with respect to the 100 sets of clauses in Table 6, and compare it to the performance of Algorithm C in Table 7.

521. [30] Experiment with the following idea, which is much simpler than the clause-purging method described in the text: “Forget a learned clause of length k with probability pk,” where p_1 ≥ p_2 ≥ p_3 ≥ ··· is a tunable sequence of probabilities.

▸ 522. [26] (Loopless shadows.) A cyclic path within the cube P_3 □ P_3 □ P_3 is shown here, together with the three “shadows” that appear when it is projected onto each coordinate plane. Notice that the shadow at the bottom contains a loop, but the other two shadows do not. Does this cube contain a cycle whose three shadows are entirely without loops? Use SAT technology to find out.

523. [30] Prove that, for any m or n, no cycle of the graph Pm □ Pn □ P_2 has loopless shadows.

▸ 524. [22] Find all Hamiltonian paths of the cube P_3 □ P_3 □ P_3 that have loopless shadows.

▸ 525. [40] Find the most difficult 3SAT problem you can that has at most 100 variables.

526. [M25] (David S. Johnson, 1974.) If F has m clauses, all of size ≥ k, prove that some assignment leaves at most m/2k clauses unsatisfied.

Behold once more with serious labor here
Haue I refurnisht out this little frame,
Repaird some parts, defectiue here and there,
And passages new added to the same.

— SAMUEL DANIEL, Certaine small Workes Heretofore Divulged (1607)

Answers to Exercises

It isn’t that they can’t see the solution.
It is that they can’t see the problem.

— G. K. CHESTERTON, The Scandal of Father Brown (1935)

Notes on the Exercises

1. A moderately easy problem for a mathematically inclined reader.

2. The author will reward you if you are first to report an error in the statement of an exercise or in its answer, assuming that he or she is suitably sagacious.

3. See H. Poincaré, Rendiconti del Circolo Matematico di Palermo 18 (1904), 45–110; R. H. Bing, Annals of Math. (2) 68 (1958), 17–37; G. Perelman, arXiv:math/0211159 [math.DG] (2002), 39 pages; 0303109 and 0307245 [math.DG] (2003), 22+7 pages.

Mathematical Preliminaries Redux

1. (a) A beats B in 5+0+5+5+0+5 cases out of 36; B beats C in 4+2+4+4+2+4; C beats A in 2+2+2+6+2+6.

(b) The unique solution, without going to more than six spots per face, is

(c) A = {Fm−2 × 1, Fm−1 × 4}, B = {Fm × 3}, C = {Fm−1 × 2, Fm−2 × 5} makes Pr(C>A)=Fm−2⁢Fm+1/Fm2; and we have Fm−2Fm+1 = Fm−1Fm− (−1)m. [Similarly, with n faces and A = {⌊n/ϕ^2⌋ × 1, ⌈n/ϕ⌉ × 4}, etc., the probabilities are 1/ϕ−O(1/n). See R. P. Savage, Jr., AMM 101 (1994), 429–436. Additional properties of nontransitive dice have been explored by J. Buhler, R. Graham, and A. Hales, AMM 125 (2018), 387–399.]

2. Let Pr(A > B) = 𝓐, Pr(B>C) = 𝓑, Pr(C>A) = 𝓒. We can assume that no x appears on more than one die; if it did, we could replace it by x + in A and x − in C (for small enough ∊) without decreasing 𝓐, 𝓑, or 𝓒. So we can list the face elements in nondecreasing order and replace each one by the name of its die; for example, the previous answer (b) yields CBBBAAAAACCCCCBBBA. Clearly AB, BC, and CA are never consecutive in an optimal arrangement of this kind: BA is always better than AB.

Suppose the sequence is Cc_1 Bb_1 Aa_1 ... Cck Bbk Aak where ci > 0 for 1 ≤ i ≤ k and bi, ai > 0 for 1 ≤ i < k. Let αi = ai/(a_1 + ··· + ak), βi = bi/(b_1 + ··· + bk), γi = ci/(c_1 + ··· + ck); then 𝓐 = α_1β_1 + α_2(β_1 + β_2)+ ···, 𝓑 = β_1γ_1 + β_2(γ_1 + γ_2)+ ···, 𝓒 = γ_2α_1 + γ_3(α_1 + α_2)+ ···. We will show that min(𝓐, 𝓑, 𝓒) ≤ 1/ϕ when the α’s, β’s, and γ’s are nonnegative real numbers; then it is < 1/ϕ when they are rational.

The key idea is that we can assume k ≤ 2 and α_2 = 0. Otherwise the following transformation leads to a shorter array without decreasing 𝓐, 𝓑, or 𝓒:

γ2′=λγ2,γ1′=γ1+γ2−γ2′,β2′=λβ2,β1′=β1+β2−β2′,α1′=α1/λ, α2′=α1+α2−α1′.

Indeed, 𝓐′ = 𝓐, 𝓒′ = 𝓒, and 𝓑′ − 𝓑 = (1 − λ)(β_1 − λβ_2)γ_2, and we can choose λ thus:

Case 1: β_1 ≥ β_2. Choose λ = α_1/(α_1 + α_2), making α2′=0.

Case 2: β_1 < β_2 and γ_1/γ_2 ≤ β_1/β_2. Choose λ = 1 + γ_1/γ_2, making γ1′=0.

Case 3: β_1 < β_2 and γ_1/γ_2 > β_1/β_2. Choose λ = 1 + β_1/β_2, making β1′=0.

Finally, then, 𝓐 = β_1, 𝓑 = 1 − β_1γ_2, 𝓒 = γ_2; they can’t all be greater than 1/ϕ.

[Similarly, with n dice, the asymptotic optimum probability pn satisfies pn=α2(n)=1−α1(n−1)α2(n)=⋅⋅⋅=1−α1(2)α2(3)=α1(2). One can show that fn(1 − pn) = 0, where fn+1(x) = fn(x)−xfn−1(x), f_0(x) = 1, f_1(x) = 1−x. Then fn(x^2) is expressible as the Chebyshev polynomial xn+1Un+1(12x); and we have pn = 1−1/(4 cos^2 π/(n+2)). See Z. Usiskin, Annals of Mathematical Statistics 35 (1964), 857–862; S. Trybu la, Zastosowania Matematyki 8 (1965), 143–156; A. Komisarski, AMM 128 (2021), 423–434.]

3. Brute force (namely a program) finds eight solutions, of which the simplest is

all with respective probabilities 1727,1627,1627. [If is also allowed, the unique solution

has the property that every roll has exactly one die below the average and two above, with each of A, B, C equally likely to be below; hence all three probabilities are 2/3. See J. Moraleda and D. G. Stork, College Mathematics Journal 43 (2012), 152–159.]

4. (a) The permutation (1 2 3 4)(5 6) takes A → B → C → D → A. So B versus C is like A versus B, etc. Also Pr(A beats C) = Pr(C beats A) = Pr(B beats D) = Pr(D beats B) =288720; Pr(A and C tie) = Pr(B and D tie) =144720.

(b) Assume by symmetry the players are A, B, C. Then the bingoers are (A, B, C, AB, AC, BC, ABC) with respective probabilities (168, 216, 168, 48, 72, 36, 12)/720.

(c) It’s (A, AB, AC, ABC, ABCD) with probabilities (120, 24, 48, 12, 0)/720.

5. (a) If Ak = 1001 with probability .99, otherwise Ak = 0, but Bk = 1000 always, then P_1000 = .99^1000 ≈ .000043. (This example gives the smallest possible P_1000, because Pr((A1−B1)+⋅⋅⋅+(An−Bn)>0≥Pr([A1>B1]...[An>Bn])=P1n.)

(b) Let E = q_0 + q_2 + q_4 + ··· ≈ 0.67915 be the probability that B = 0. Then Pr(A>B)=Σk=0∞⁢  q2k (E+Σj=0k−1⁢ q2j+1)≈.47402; Pr(A<B)=Σk=0∞⁢  q2k+1 (1−E+Σj=0k⁢ q2j)≈.30807; and Pr(A = B) = Pr(A = B = 0) = E(1 − E) ≈ .21790 is also the probability that AB > 0.

(c) During the first nk rounds, the probability that either Alice or Bob has scored more than mk is at most nk(qk+1 + qk+2 + ··· ) = O(2−k); and the probability that neither has ever scored mk is (1 − qk)nk < exp(−qknk) = exp(−2k/D). Also mk > nkmk−1 when k > 1. Thus Alice “quite surely” wins when k is even, but loses when k is odd, as k → ∞. [The American Statistician 43 (1989), 277–278.]

6. The probability that Xj = 1 is clearly p_1 = 1/(n−1); hence Xj = 0 with probability p_0 = (n − 2)/(n − 1). And the probability that Xi = Xj = 1 when i < j is p12. Thus (see exercise 20), (Xi, Xj) will equal (0, 1), (1, 0), or (0, 0) with the correct probabilities p_0p_1, p_1p_0, p_0p_0. But Xi = Xj = Xk = 1 with probability 0 when i < j < k.

For 3-wise independence let Pr(X_1 ... Xn = x_1 ... xn) = ax1+···+xn/(n−2)3, where a0=2(n−23), a1=(n−22), a_3 = 1, otherwise aj = 0.

7. Let fm(n)=Σj=0m(nj)(−1)j(n+1−m)m−j, and define probabilities ia aj fk−j(n−j) as in answer 6. (In partiular, we have f_0(n) = 1, f_1(n) = 0, f2(n)=(n−12), f3(n)=2(n−23), f4(n)=3(n−34)+(n−32)2.) This definition is valid if we can prove that fm(n) ≥ 0 for n ≥ m, because of the identity Σj(nj) fm−j (n−j)=(n+1−m)m.

To prove that inequality, Schulte-Geers notes (see CMath (5.19)) that fm(n)=Σk=0m(m−nk)(n−m)m−k=Σk=0m(n−m−1+kk)(−1)k(n−m)m−k; these terms pair up nicely to yield Σk=0m−1⁢ k(n−m−1+kk+1)(n−m)m−k−1[k⁢  even]+(n−1m)⁢ [m  even].

8. If 0 < k < n, the probability that k of the components have any particular setting is 1/2k, because the remaining components have even parity as often as odd parity. So there’s (n − 1)-wise independence, but not n-wise.

9. Give probability 1/2 to 0 ... 0 and 1 ... 1; all other vectors have probability 0.

10. If n > p we have Xp+1 = X_1, so there’s no independence. Otherwise, if m < n ≤ p, there’s m-wise independence because any m vectors (1,j,...,jm−1) are linearly independent modulo p (they’re columns of Vandermonde’s matrix, exercise 1.2.3–37); but the X’s are dependent (m + 1)-wise, because a polynomial of degree m cannot have m + 1 different roots. If m ≥ n and n ≤ p there is complete independence.

Instead of working mod p, we could use any finite field in this construction.

11. We can assume that n = 1, because (X_1 + ··· + Xn)/n and (Xn+1 + ··· + X2n)/n are independent random variables with the same discrete distribution. Then Pr(|X_1 + X_2 − 2α| ≤ 2|X_1 − α|) ≥ Pr(|X_1 − α| + |X_2 − α| ≤ 2|X_1 − α|) = Pr(|X_2 − α| ≤ |X_1 − α|) = (1 + Pr(X_1 = X_2))/2 > 1/2. [This exercise was suggested by T. M. Cover.]

12. Let w = Pr(A and B), x=Pr(A⁢ and B¯), y = Pr(Ā and B), z=Pr(A¯⁢ and B¯). All five statements are equivalent to wz > xy, or to |wxyz|>0, or to “A and B are strictly positively correlated” (see exercise 61). [This exercise was suggested by E. Georgiadis.]

13. False in many cases. For example, take Pr(A¯⁢  and  B¯⁢ ⁢ and  C¯)=Pr(A¯⁢  ⁢ and  B⁢ and  C¯)=0, Pr(A and B and C) = 2/7, and all other joint probabilities 1/7.

14. Induction on n. [Philosophical Transactions 53 (1763), 370–418, proof of Prop. 6.]

15. If Pr(C) > 0, this is the chain rule, conditional on C. But if Pr(C) = 0, it’s false by our conventions, unless A and B are independent.

16. If and only if Pr(A¯∩B∩C)=0≠Pr(B) or Pr(A¯∩C)=0

17. 4/51, because four of the cards other than Q♠ are aces.

18. Since (M − X)(X − m) ≥ 0, we have (M E X) − (E X^2) + (m E X) − mM ≥ 0. [See C. Davis and R. Bhatia, AMM 107 (2000), 353–356, for generalizations.]

19. (a) The binary values of Pr(Xn = 1) = E Xn for n = 0, 1, 2, ... , are respectively (.0101010101010101 ... )2, (.0011001100110011 ... )2, (.0000111100001111 ... )2, ... ; thus they’re the complemented reflections of the “magic masks” 7.1.3–(47). The answer is therefore (22n − 1)/(22n + 1 − 1) = 1/(22n + 1).

(b) Pr(X0X1⁢ ...⁢Xn−1=x0x1⁢...⁢xn−1)=2(x¯n−1...⁢x¯n−1...x¯1x¯0)2/(22n−1)  can be “read off” from the magic masks by ANDing and complementing. [See E. Lukacs, Characteristic Functions (1960), 119, for related theory.]

(c) The infinite sum S is well defined because Pr(S = ∞) = 0. Its expectation E S=Σn=0∞1/(22n+1)≈0.59606 corresponds to the case z = 1/2 in answer 7.1.3– 41(c). By independence, var(S)=Σn=0∞ var(Xn)=Σn=0∞22n/(22n+1)2≈0.44148.

(d) The parity number E R = (.0110100110010110 ... )2 has the decimal value

0.41245 40336 40107 59778 33613 68258 45528 30895−,
and can be shown to equal 12−14P where P=Πk=0∞(1−1/22k) [R. W. Gosper and R. Schroeppel, MIT AI Laboratory Memo 239 (29 February 1972), Hack 122], which is transcendental [K. Mahler, Mathematische Annalen 101 (1929), 342–366; 103 (1930), 532]. (Furthermore it turns out that 1/P−1/2=Σk=0∞1/Πj=0k−1(22j−1).) Since R is binary, var(R) = (E R)(1 − E R) ≈ 0.242336.

(e) Zero (because π is irrational, hence p_0 + p_1 + ··· = ∞). However, if we ask the analogous question for Euler’s constant γ instead of π, nobody knows the answer.

(f) E Yn = 2E Xn; in fact, Pr(Y_0Y_1Y_2 ... = x_0x_1x_2 ... ), for any infinite string x_0x_1x_2 ... , is equal to 2 Pr(X_0X_1X_2 ... = x_0x_1x_2 ... ) mod 1, because we shift the binary representation one place to the left (and drop any carry). Thus in particular, E YmYn=2⁢  E XmXn=12⁢ E Ym E Yn when m ≠ n; Ym and Yn are negatively correlated because covar (Ym,Yn)=−12⁢ E Ym⁢ E Yn.

(g) Clearly E T = 2E S. Also E T^2 = 2E S^2, because E YmYn = 2E XmXn for all m and n. So var(T) = 2(var(S)+(E S)2) − (2 E S)2 = 2 var(S) − 2(E S)2 ≈ 0.17237.

20. Let pj = E Xj. We must prove, for example, that E(X_1(1 − X_2)(1 − X_3)X_4) = p_1(1 − p_2)(1 − p_3)p_4 when k ≥ 4. But this is E(X_1X_4 − X_1X_2X_4 − X_1X_3X_4 + X_1X_2X_3X_4) = p_1p_4 − p_1p_2p_4 − p_1p_3p_4 + p_1p_2p_3p_4.

21. From the previous exercise we know that they can’t both be binary. Let X be binary and Y ternary, taking each of the values (0, 0), (0, 2), (1, 0), (1, 1), (1, 2) with probability 1/5. Then E XY = E X = 3/5 and E Y = 1; Pr(X = 0) Pr(Y = 1) = 2/25 ≠ 0.

22. By (8) we have Pr(A_1 ∪ ··· ∪ An) = E[A_1 ∪ ··· ∪ An] = E max([A_1],..., [An]) ≤ E([A_1]+ ··· +[An]) = E[A_1 ]+ ··· +E[An] = Pr(A_1)+ ··· +Pr(An).

23. The hinted probability is Pr(Xs = 0 and X_1 + ··· + Xs−1 = s − r), so it equals (s−1s−r)ps−r(1−p)r. To get Bm,n(p), sum it for r = n − m and n − m ≤ s ≤ n. [For an algebraic rather than probabilistic/combinatorial proof, see CMath, exercise 8.17.]

24. (a) The derivative of Bm,n(x)=Σk=0m(nk)xk (1−x)n−k is

Bm,n′(x)=Σk=1m(nk) kxk−1(1−x) n−k−Σk=0m(nk) (n−k) xk⁢ (1−x)n−1−k=n(Σk=0m−1⁢    (n−1k)xk⁢ (1−x) n−1−k−Σk=0m(n−1k)xk(1−x)n−1−k)=n(n−1m)xm(1−x)n−1−m.

[See Karl Pearson, Biometrika 16 (1924), 202–203.]

(b) The hint, which says that ∫0a/(a+b+1) xa(1−x)bdx<∫a/(a+b+1)1 xa(1−x)bdx when 0 ≤ a ≤ b, will prove that 1 − Bm,n(m/n) < Bm,n(m/n). If a > 0 it suffices to show that ∫0a/(a+b) xa(1−x)bdx≤∫a/(a+b)1 xa(1−x)bdx, because we have ∫0a/(a+b+1)<∫0a/(a+b)≤∫a/(a+b)1<∫a/(a+b+1)1. Let x = (a−∊)/(a+b), and observe that (a−∊)a(b+∊)b is less than or equal to (a + ∊)a(b − ∊)b for 0 ≤ ∊ ≤ a, because the quantity

(a−∈a+∈)a=ea(ln(1−∈/a)−ln(1+∈/a))=exp(−2∈(1+∈23a2+∈45a4+⋅⋅⋅))

is nondecreasing when a increases.

(c) Let tk=(nk)mk(n−m)n−k. When m ≥ n/2 we can show that 1−Bm,n(m/n)=Σk>m⁢tk/nn<Bm,n(m/n)=Σk=0m⁢ tk/nn, because tm+d < tm+1−d for 1 ≤ d ≤ n − m. For if rd = tm+d/tm+1−d, we have r_1 = m/(m + 1) < 1; also
rd+1rd=(n−m+d) (n−m−d)m2(m+1+d) (m+1−d) (n−m)2<1,
because ((m+1)2 − d^2)(n−m)2 − ((n−m)2 −d^2)m^2 = (2m+1)(n−m)2 +(2m−n)nd^2.

[Peter Neumann proved in Wissenschaftliche Zeitschrift der Technischen Universit¨at Dresden 15 (1966), 223–226, that m is the median. The argument in part (c) is due to Nick Lord, in The Mathematical Gazette 94 (2010), 331–332. See also S. Janson, Statistics and Probability Letters 171 (2021) 109020, 10 pages.]

25. (a) ((nk))−((nk+1)) is ∑pIqj (qt/(n − k) − pt/(k +1)), summed over all partitions of {1,...,n} into disjoint sets I ∪ J ∪ {t}, where |I| = k, |J| = n − k − 1, pI = ∏i∊I pi, qJ = ∏j∊J qj. And qt/(n − k) − pt/(k +1) ≥ 0 ⇔ pt ≤ (k +1)/(n + 1).

(b) Given p_1, ... , pn−1, the quantity ((nk)) is maximized when pn = p, by (a). The same argument applies symmetrically to all indices j.

26. The inequality is equivalent to rn,k2≥rn,k−1rn,k+1, which was stated without proof on pages 242–245 of Newton’s Arithmetica Universalis (1707), then finally proved by Sylvester many years later [Proc. London Math. Soc. 1 (1865), 1–16]. We have nrn,k = kpnrn−1,k−1 +(n − k)qnrn−1,k; hence n2(rn,k2−rn,k−1rn,k+1)=(pnrn−1,k−1−qnrn−1,k)2+(k2−1)pn2A+(k−1) (n−1−k)pnqn⁢ B+((n−k)2−1)qn2C, where A=rn−1,k−12−rn−1,k−2rn−1,k, B=rn−1,k−1rn−1,k−rn−1,k−2rn−1,k+1 and C=rn−1,k2−rn−1,k−1rn−1,k+1 are nonnegative, by induction on n.

27. Σk=0m((nk))=Σk=0m((n−m−1+kk))(1−pn−m+k), by the same argument as before.

28. (a) ((nk))=((n−2k))A+((n−2k−1))B+((n−2k−2)) C and E g(X)=Σk=0n−2((n−2k))hk, where A = (1 − pn−1)(1 − pn), C = pn−1pn, B = 1 − A − C, and hk = Ag(k)+ Bg(k +1)+ Cg(k + 2). If the pj’s aren’t all equal, we may assume that pn−1 < p < pn. Setting pn−1′=pn−1+∈ and pn′=pn−∈, where ∊ = min(pn − p, p − pn−1), changes A, B, C to A′ = A + δ, B′ = B − 2δ, C′ = C + δ, where δ = (pn − p)(p − pn−1); hence hk changes to hk′=hk+δ(g(k)−2g(k+1)+g(k+2)). Convex functions satisfy g(k) − 2g(k + 1) + g(k + 2) ≥ 0, by (19) with x = k and y = k + 2; hence we can permute the p’s and repeat this transformation until pj = p for 1 ≤ j ≤ n.

(b) Suppose E g(X) is maximum, and that r of the p’s are 0 and s of them are 1. Let a satisfy (n − r − s)a + s = np and assume that 0 < pn−1 < a < pn < 1. As in part (a) we can write E g(X) = αA + βB + γC for some coefficients α, β, γ.

If α−2β + γ > 0, the transformation in (a) (but with a in place of p) would increase E g(X). And if α − 2β + γ < 0, we could increase it with a similar transformation, using δ = − min(pn−1, 1 − pn). Therefore α − 2β + γ = 0; and we can repeat the transformation of (a) until every pj is 0, 1, or a.

(c) Since Σk=0m((nk))=0 when s > m, we may assume that s ≤ m, hence r+s < n. For this function g(k) = [0 ≤ k ≤ m] we have α−2β+γ=((n−2m))−((n−2m−1)). This difference cannot be positive if the choice of {p_1,...,pn} is optimum; in particular we cannot have s = m. If r > 0 we can make pn−1 = 0 and pn = a, so that ((n−2m))=(n−r−s−1m−s)am−s(1−a)n−r−1−m and ((n−2m−1))=(n−r−s−1m−1−s)am−1−s(1−a)n−r−m. But then the ratio ((n−2m))/((n−2m−1))=(n−r−m)a/((m−s)(1−a)) exceeds 1; hence r = 0.

Similarly s > 0 we can set (pn−1,pn) = (a, 1), getting the ratio ((n−2m))/((n−2m−1))=(n−r−m)a/((m−s+1)(1−a))≥1. In this case ((n−2m))=((n−2m−1)) if and only if np = m + 1; we can transform without changing E g(X), until s = 0 and each pj = p.

[Reference: Annals of Mathematical Statistics 27 (1956), 713–721. The coefficients ((nk)) also have many other important properties; see exercise 7.2.1.5–63 and the survey by J. Pitman in J. Combinatorial Theory A77 (1997), 279–303.]

29. The result is obvious when m = 0 or n; and there’s a direct proof when m = n − 1: Bn−1,n(p) = 1 − pn ≥ (1 − p)n/((1 − p)n + p) because p − npn + (n − 1)pn+1 = p(1 − p)(1 + p + ··· + pn−1 − pn−1n) ≥ 0. The result is also clear when p = 0 or 1.

If p = (m + 1)/n we have Rm,n(p) = ((1 − p)(m + 1)/((1 − p)m + 1))n−m = ((n − m − 1)/(n − m))n−m. So if m > 0 and p^=m/(n−1), we can apply exercise 28(c) with p1=⋅⋅⋅=pn−1=p^ and pn = 1:

Bm,n (p)≥Σk=0m((nk))=Σk=0m(n−1k−1)p^k−1(1−p^)n−k=Bm−1,n−1(p^).

When 1 ≤ m < n − 1, let Qm,n(p) = Bm,n(p) − Rm,n(p). The derivative

Qm,n′(p)=(n−m)(nm)(1−p)n−m−1(A−F(p))/((1−p)m+1)n−m+1,
where A=(m+1)n−m/(nm)>1 and F (p) = pm((1 − p)m +1)n−m+1, begins positive at p = 0, eventually becomes negative but then is positive again at p = 1. (Notice that F (0) = 0, and F (p) increases dramatically until p = (m +1)/(n + 1); then it decreases to F (1) = 1.) The facts that Qm,n(m+1n)≥0=Qm,n(0)=Qm,n(1) now complete the proof, because Qm,n′(p) changes sign only once in [0..m+1n]. [Annals of Mathematical Statistics 36 (1965), 1272–1278.]

30. (a) Pr(Xk = 0) = n/(n + 1); hence p = nn/(n +1)n > 1/e ≈ 0.368.

(b) (Solution by J. H. Elton.) Let pkm = Pr(Xk = m). Assume that these probabilities are fixed for 1 ≤ k < n, and let xm = pnm. Then x_0 = x_2 + 2x_3 + 3x_4 +··· ; we want to minimize p=Σm=1∞(Am+(m−1)A0)xm in nonnegative variables x_1, x_2, ... , where Am = Pr(X_1 +···+ Xn−1 ≤ n−m), subject to the condition Σm=1∞mxm=1. Since all coefficients of p are nonnegative, the minimum is achieved when all xm for m ≥ 1 are zero except for one value m = mn, which minimizes (Am +(m − 1)A_0)/m. And mn ≤ n + 1, because Am = 0 whenever m > n. Similarly m_1, ... , mn−1 also exist.

(c) (Solution by E. Schulte-Geers.) Letting m_1 = ··· = mn = t ≤ n + 1, we want to minimize B⌊n/t⌋,n(1/t). The inequality of Samuels in exercise 29 implies that

Bm,n(p)≥(1−1f(m,n,p)+1)n for p≤m+1n, where f(m,n,p)=(m+1)(1−p)n(n−m)p,
because we can set x = ((1 − p)m +1)/((1 − p)(m + 1)) in the arithmetic–geometric mean inequality xn−m ≤ ((n − m)x + m)n/nn. Now 1/t ≤ (⌊n/t⌋ + 1)/(n + 1) and f(⌊n/t⌋,n, 1/t) ≥ n; hence B⌊n/t⌋,n(1/t) ≥ nn/(n +1)n.

[Peter Winkler called this the “gumball machine problem” in CACM 52, 8 (August 2009), 104–105. J. H. Elton has verified that the joint distributions in (a) are optimum when n ≤ 20; see arXiv:0908.3528 [math.PR] (2009), 7 pages. Do those distributions in fact minimize p for all n? Uriel Feige has conjectured more generally that we have Pr(X_1 + ··· + Xn < n + 1/(e−1)) ≥ 1/e whenever X_1, ... , Xn are independent nonnegative random variables with E Xk ≤ 1; see SICOMP 35 (2006), 964–984.]

31. This result is immediate because Pr(f([A_1],..., [An])) = E f([A_1],..., [An]). But a more detailed, lower-level proof will be helpful with respect to exercise 32.

Suppose, for example, that n = 4. The reliability polynomial is the sum of the reliability polynomials for the minterms of f; so it suffices to show that the result is true for functions like x1∧x¯2∧x¯3∧x4=x1(1−x2)(1−x3)x4. And it’s clear that Pr(A1∩A¯2∩A¯3∩A4)=Pr(A1∩A¯2∩A4)−Pr(A1∩A¯2∩A3∩A4)=π14−π124−π134+π1234. (See exercise 7.1.1–12; also recall the inclusion-exclusion principle.)

32. The 2n minterm probabilities in the previous answer must all be nonnegative, and they must sum to 1. We’ve already stipulated that π∅ = 1, so the sum-to-1 condition is automatically satisfied. (The condition stated in the exercise when I ⊆ J is necessary but not sufficient; for example, π_12 must be ≥ π_1 + π_2 − 1.)

33. The three events (X, Y) = (1, 0), (0, 1), (1, 1) occur with probabilities p, q, r, respectively. The value of E(X | Y) is 1, r/(q + r), r/(q + r) in those cases. Hence the answer is pz + (q + r)zr/(q+r). (This example demonstrates why univariate generating functions are not used in the study of conditional random variables such as E(X | Y). But we do have the simple formula E(X|Y=k)=([zk]∂∂wG(1,z))/([zk]G(1,z)).)

34. The right-hand side is

ΣwE(X|Y)Pr(w)=ΣwPr(w)Σw′X(w′)Pr(w′)[Y(w′)=Y(w)]/Pr(Y=Y(w))=ΣwPr(w)Σw′X(w′)Pr(w′)[Y(w′)=Y(w)]/Pr(Y=Y(w′))=Σw′X(w′)Pr(w′)ΣwPr(w)[Y(w)=Y(w′)]/Pr(Y=Y(w′)).

35. Part (b) is false. If, for instance, X and Y are independent random bits and Z = X, we have E(X|Y)=12 and E(12|Z)=12≠X=E(X|Z). The correct formula instead of (b) is

E(E(X|Y,Z)|Z)=E(X|Z).(*)

This is (12) in the probability spaces conditioned by Z, and it is the crucial identity that underlies exercise 91. Part (a) is true because it is the case Y = Z of (∗).

36. (a) f(X); (b) E(f(Y)g(X)), generalizing (12). Proof: E(f(Y)E(g(X) | Y)) = Σy f(y)E(g(X) | Y = y) Pr(Y = y) = Σx,y f(y)g(x)Pr(X = x, Y = y) = E(f(Y)g(X)).

37. If we’re given the values of X_1, ... , Xk−1, the value of Xk is equally likely to be any of the n + 1 − k values in {1,...,n}\{X_1,...,Xk−1}. Hence its average value is (1 + ··· + n − X_1 −···− Xk−1)/(n + 1 − k). We conclude that E(Xk | X_1,...,Xk−1) = (n(n + 1)/2 − X_1 −···− Xk−1)/(n + 1 − k). [Incidentally, the sequence Z_0, Z_1, ... , defined by Zj = (n + j)X_1 +(n + j − 2)X_2 + ··· + (n − j)Xj+1 − (j + 1)n(n + 1)/2 for 0 ≤ j < n and Zj = Zn−1 for j ≥ n, is therefore a martingale.]

38. Let tm,n be the number of restricted growth strings of length m + n that begin with 01 ... (m−1). (This is the number of set partitions of {1,...,m+n} in which each of {1,...,m} appears in a different block.) The generating function Σn≥0tm,nzn/n! turns out to be exp(ez − 1 + mz); hence tm,n=Σkϖk(nk)mn−k.

Suppose M = max(X_1,...,Xk−1) + 1. Then Pr(Xk = j) = tM,n−k/tM,n+1−k for 0 ≤ j < M, and tM+1,n−k/tM,n+1−k for j = M. Hence E(Xk|X0,...,Xk−1)=((m2)tM,n−k+MtM+1,n−k)/tM,n+1−k.

39. (a) Since E(K | N = n) = pn we have E(K | N) = pN.

(b) Hence E K = E(E(K | N)) = E pN = pμ.

(c) Let pnk=Pr(N=n,K=k)=(e−μμn/n!)×(nk)pk(1−p)n−k=(e−μμkpk/k!)×f(n−k), where f(n) = (1 − p)nμn/n!. Then E(N | K = k) = Σnnpnk/Σnpnk. Since nf(n − k) = kf(n − k)+(n − k)f(n − k) and nf(n) = (1 − p)μf(n − 1), the answer is k + (1 − p)μ; hence E(N | K) = K + (1 − p)μ. [G. Grimmett and D. Stirzaker, Probability and Random Processes (Oxford: 1982), §3.7.]

40. If p = Pr(X > m), clearly E X ≤ (1 − p)m + pM. [We also get this result from (15), by taking S = {x | x ≤ m}, f(x) = M − x, s = M − m.]

41. (a) Convex when a ≥ 1 or a = 0; otherwise neither convex nor concave. (However, xa is concave when 0 < a < 1 and convex when a < 0, if we consider only positive values of x.) (b) Convex when n is even or n = 1; otherwise neither convex nor concave. (This function is ∫0xtn−1ex−tdt/(n−1)!, according to 1.2.11.3–(5); so f″(x)/x > 0 when n ≥ 3 is odd.) (c) Convex. (In fact f(|x|) is convex whenever f(z) has a power series with nonnegative coefficients, convergent for all z.) (d) Convex, provided of course that we allow f to be infinite in the definition (19).

42. We can show by induction on n that f(p_1x_1+···+pnxn) ≤ p_1f(x_1)+···+pnf(xn), when p_1, ... , pn ≥ 0 and p_1 + ··· + pn = 1, as in exercise 6.2.2–36. The general result follows by taking limits as n → ∞. [The quantity p_1x_1 + ··· + pnxn is called a “convex combination” of {x_1,...,xn}; similarly, E X is a convex combination of X values. Jensen actually began his study by assuming only the case p=q=12 of (19).]

43. f(E X) = f(E(E(X | Y ))) ≤ E(f(E(X | Y))) ≤ E(E f(X) | Y) = E f(X). [S. M. Ross, Probability Models for Computer Science (2002), Lemma 3.2.1.]

44. The function f(xy) is convex in y for any fixed x. Therefore g(y) = E f(Xy) is convex in y: It’s a convex combination of convex functions. Also g(y) ≥ f(E Xy) = f(0) = g(0) by (20). Hence 0 ≤ a ≤ b implies g(0) ≤ g(a) ≤ g(b) by convexity of g. [S. Boyd and L. Vandenberghe, Convex Optimization (2004), exercise 3.10.]

45. Pr(X > 0) = Pr(|X| ≥ 1); set m = 1 in (16).

46. E X^2 ≥ (E X)2 in any probability distribution, by Jensen’s inequality, because squaring is convex. We can also prove it directly, since E X^2 − (E X)2 = E(X − E X)2.

47. We always have Y ≥ X and Y^2 ≤ X^2. (Consequently (22) yields Pr(X > 0) = Pr(Y > 0) ≥ (E Y)2/(E Y^2) ≥ (E X)2/(E X^2) when E X ≥ 0.)

48. Pr(a−X1−⋅⋅⋅−Xn>0)≥a2/(a2+σ12+...+σn2), by exercise 47. [This inequality was also known to Chebyshev; see J. Math. Pures et Appl. (2) 19 (1874), 157–160. In the special case n = 1 it is equivalent to “Cantelli’s inequality,”

Pr(X≥EX+a)≤var(X)/(var(X)+a2),for a≥0;
see Atti del Congresso Internazionale dei Matematici 6 (Bologna: 1928), 47–59, §6–§7.]

49. Pr(X = 0) = 1−Pr(X > 0) ≤ (E X^2−(E X)2)/ E X^2 ≤ (E X^2−(E X)2)/(E X)2 = (E X^2)/(E X)2 − 1. [Some authors call this inequality the “second moment principle,” but it is strictly weaker than (22).]

50. (a) Let Yj = Xj/X if Xj > 0, otherwise Yj = 0. Then Y_1 + ··· + Ym = [X > 0]. Hence Pr(X>0)=Σj=1mEYj; and EYj = E(Xj/X|Xj>0). Pr(Xj>0). [This identity, which requires only that Xj ≥ 0, is elementary yet nonlinear, so it apparently lay undiscovered for many years. See D. Aldous, Discrete Math. 76 (1989), 168.]

(b) Since Xj ∊ {0, 1}, we have Pr(Xj > 0) = E Xj = pj; and E(Xj/X | Xj > 0) = E(Xj/X | Xj =1) = E(1/X | Xj = 1) ≥ 1/ E(X | Xj =1).

(c) Pr(XJ=1)=Σj=1mPr(J=j⁢ and Xj=1)=Σj=1mpj/m=EX/m. Hence Pr(J = j | Xj = 1) = Pr(J = j and Xj = 1)/ Pr(Xj = 1) = (pj/m)/(E X/m) = pj/E X.

(d) Since J is independent we have tj = E(X | J = j and Xj = 1) = E(X | Xj = 1).

(e) The right side is (EX)Σj=1m(pj/EX)/tj≥(EX)/Σj=1m(pj/EX)tj.

51. If g(q_1,...,qm) = 1 − f(p_1,...,pm) is the dual of f, where qj = 1 − pj, a lower bound on g gives an upper bound on f. For example, when f is x_1x_2x_3 ∨ x_2x_3x_4 ∨ x_4x_5, f¯ is x¯1x¯4+x¯2x¯4+x¯3x¯4+x¯2x¯5+x¯3x¯5. So the inequality (24) gives g(q_1,...,q_5) ≥ q_1q_4/(1 + q_2 + q_3 + q_2q_5 + q_3q_5) + q_2q_4/(q_1 + 1 + q_3 + q_5 + q_3q_5) + q_3q_4/(q_1 + q_2 + 1 + q_2q_5 + q_5)+ q_2q_5/(q_1q_4 + q_4 + q_3q_4 + 1 + q_3) + q_3q_5/(q_1q_4 + q_2q_4 + q_4 + q_2 + 1). In particular, g(.1,...,.1) > 0.039 and f(.9,...,.9) < 0.961.

52. (nk)pk/Σj=0k(kj)(n−kj)pj.

53. f(p_1,...,p_6) ≥ p_1p_2(1−p_3)/(1+p_4p_5(1−p_6))+···+p_6p_1(1−p_2)/(1+p_3p_4(1−p_5)). Monotonicity is not required when applying this method, nor need the implicants be prime. The result is exact when the implicants are disjoint.

54. (a) Pr(X>0)≤EX=(n3)p3, because E Xuvw = p^3 for all u < v < w.

(b) Pr(X > 0) ≥ (E X)2/(E X^2), where the numerator is the square of (a) and the denominator can be shown to be (n3)p3+12(n4)p5+30(n5)p6+20(n6)p6. For example, the expansion of X^2 contains 12 terms of the form XuvwXuvw′ with u < v < w < w′, and each of those terms has expected value p^5.

55. A BDD for the corresponding Boolean function of (102)=45 variables has about 1.4 million nodes, and allows us to evaluate the true probability (1 − p)45G(p/(1 − p)) exactly, where G(z) is the corresponding generating function (see exercise 7.1.4–25). The results are: (a) 30/37 ≈ .811 < 35165158461687/2^45 ≈ .999 < 15; (b) 10/109 ≈ .092 < 4180246784470862526910349589019919032987399/(4 × 10^43) ≈ .105 < .12.

56. The upper bound is μ = λ^3/6; the lower bound divides this by 1 + μ. [The exact asymptotic value can be obtained using the principle of inclusion and exclusion and its “bracketing” property, as in Eq. 7.2.1.4–(48); the result is 1 − e−μ. See P. Erdős and A. Rényi, Magyar Tudományos Akadémia Mat. Kut. Int. Közl. 5 (1960), 17–61, §3.]

57. To compute E(X | Xuvw = 1) we sum Pr(Xu′v′w′ | Xuvw = 1) over all (n3) choices of u′ < v′ < w′. If {u′,v′,w′}∩{u, v, w} has t elements, this probability is p3−t(t−1)/2; and there are (3t)(n−33−t) such cases. Consequently we get

Pr(X>0)≥(n3)p3/((n−33)p3+3(n−32)p3+3(n−31)p2+(n−30)p0).

[In this problem the lower bound turns out to be the same using either inequality; but the derivation here was easier.]

58. Pr(X>0)≤(nk)pk(k−1)/2. The lower bound, using the conditional expectation inequality as in the previous answer, divides this by Σt=0k(kt)(n−kk−t)pk(k−1)/2−t(t−1)/2.

59. (a) It suffices to prove that a_0b_1 + a_1b_0 ≤ c_0d_1 + c_1d_0. The key observation is that c_1d_0(c_0d_1 + c_1d_0 − a_0b_1 − a_1b_0) = (c_1d_0 − a_0b_1)(c_1d_0 − a_1b_0)+(c_0c_1d_0d_1 − a_0a_1b_0b_1). Thus the result holds when c_1d_0 ≠ 0; and if c_1d_0 = 0 we have a_0b_1 + a_1b_0 = 0.

All four hypotheses hold with equality when a_0 = b_0 = d_0 = 0 and the other variables are 1, yet the conclusion is that 1 ≤ 2. Conversely, when b_1 = c_1 = 2 and the other variables are 1, we have a_1b_0 < c_1d_0 but conclude only that 6 ≤ 6.

(b) Let Al = Σ {a2j+l | 0 ≤ j < 2n−1} for l = 0 and l = 1, and define Bl, Cl, Dl Similarly from b2j+l, c2j+l, d2j+l. The hypotheses for j mod 2 = l and k mod 2 = m prove that AlBm ≤ Cl|mDl&m, by induction on n. Hence, by part (a), we have the desired inequality (A_0 + A_1)(B_0 + B_1) ≤ (C_0 + C_1)(D_0 + D_1). [This result is due to R. Ahlswede and D. E. Daykin, Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete 43 (1978), 183–185, who stated it in the language of the next exercise.]

(c) Now let An = a_0 + ··· + a2n−1, and define Bn, Cn, Dn similarly. If A∞B∞ > C∞D∞, we’ll have AnBn > C∞D∞ for some n. But C∞D∞ ≥ CnDn, contra (b).

[In fact much more is true: We have Σvj+vk=najbk ≤ Σvj+vk=ncjdk, for all n. See A. Björner, Combinatorica 31 (2011), 151–164; D. Christofides, arXiv:0909.5137 [math.CO] (2009), 6 pages.]

60. (a) We can consider each set to be a subset of the nonnegative integers. Let α¯(S)=α(S)[S∈F],β¯(S)=β(S)[S∈G],γ¯(S)=γ(S)[S∈F⊔G],δ¯(S)=δ(S)[S∈F⊓G]; then α¯(℘)=α(F),β¯(℘)=β(G),γ¯(℘)=γ(F⊔G), and δ¯(℘)=δ(F⊓G), where ℘ is the family of all possible subsets. Since any set S of nonnegative integers can be encoded in the usual way as the binary number s = Σj∊S2j, the desired result follows from the four functions theorem if we let as=α¯(S),bs=β¯(S),cs=γ¯(S),ds=δ¯(S).

(b) Let α(S) = β (S) = γ (S) = δ(S) = 1 for all sets (S).

61. (a) In the hinted case we can let α(S) = f(S)μ(S), β(S) = g(S)μ(S), γ(S) = f(S)g(S)μ(S), δ(S) = μ(S); the four functions theorem yields the result. The general case follows because we have E(fg)−E(f)E(g)=E(f^g^)−E(f^)E(g^), where f^(S)=f(S)−f(∅) and g^(S)=g(S)−g(∅). [See Commun. Math. Physics 22 (1971), 89–103.]

(b) Changing f(S) to θf(S) and g(S) to ϕg(S) changes E(fg) − E(f)E(g) to θϕ(E(fg) − E(f)E(g)), for all real numbers θ and ϕ.

(c) If S and T are supported, then R = S ∩ T and U = S ∪ T are supported. Furthermore we can write S = R ∪ {s_1,...,sk} and T = R ∪ {t_1,...,tl} whre the sets Si = R ∪ {s_1,...,si} and Tj = R ∪ {t_1,...,tj} are supported, as are their unions Ui,j = Si ∪ Tj, for 0 ≤ i ≤ k and 0 ≤ j ≤ l. By (iii) we know that μ(Ui+1,j)/μ(Ui,j) ≤ μ(Ui+1,j+1)/μ(Ui,j+1) when 0 ≤ i < k and 0 ≤ j < l. Multiplying these inequalities for 0 ≤ i < k, we obtain μ(Uk,l)/μ(U0,j) ≤ μ(Uk,j+1)/μ(U0,j+1). Hence μ(S)/μ(R) = μ(Uk,0)/μ(U0,0) ≤ μ(Uk,l)/μ(U0,l) = μ(U)/μ(T).

(d) In fact, equality holds, because [j ∊ S] + [j ∊ T] = [j ∊ S ∪ T] + [j ∊ S ∩ T]. [Note: Random variables with this distribution are often confusingly called “Poisson trials,” a term that conflicts with the (quite different) Poisson distribution of exercise 39.]

(e) Choose c in the following examples so that ΣS μ(S) = 1. In each case the supported sets are subsets of U = {1,...,m}. (i) Let μ(S) = cr_1r_2···r|S|, where 0 < r_1 ≤ ··· ≤ rm. (ii) Let μ(S) = cpj when S = {1,...,j} and 1 ≤ j ≤ m, otherwise μ(S) = 0. (If p_1 = ··· = pm in this case, the FKG inequality reduces to Chebyshev’s monotonic inequality of exercise 1.2.3–31.) (iii) Let
μ(S) = cμ_1 (S ∩ U_1)μ_2(S ∩ U_2)...μk(S ∩ Uk),
where each μj is a distribution on the subsets of Uj ⊆ U that satisfies (**). The subuniverses U_1, . . . , Uk needn’t be disjoint. (iv) Let μ(S) = ce−f(S), where f is a submodular set function on the supported subsets of U: f(S ∪ T) + f(S ∩ T) ≤ f(S) + f(T) whenever f(S) and f(T) are defined. (See Section 7.6.)

62. A Boolean function is essentially a set function whose values are 0 or 1. In general, under the Bernoulli distribution or any other distribution that satisfies the condition of exercise 61, the FKG inequality implies that any monotone increasing Boolean function is positively correlated with any other monotone increasing Boolean function, but negatively correlated with any monotone decreasing Boolean function. In this case, f is monotone increasing but g is monotone decreasing: Adding an edge doesn’t disconnect a graph; deleting an edge doesn’t invalidate a 4-coloring.

(Notice that when f is a Boolean function, E f is the probability that f is true under the given distribution. The fact that covar(f, g) ≤ 0 in such a case is equivalent to saying that the conditional probability Pr(f | g) is ≤ Pr(f).)

63. If ω is the event ‘Z_0 = a and Z_1 = b’, we have Z_0(ω) = a and E(Z_1 | Z_0)(ω) = (pa_1 + 2pa_2)/(pa_0 + pa_1 + pa_2). Hence p_01 = p_02 = p_20 = p_21 = 0, and p_10 = p_12. Those conditions are necessary and sufficient for E(Z_1 | Z_0) = Z_0.

64. (a) No. Consider the probability space consisting of just three events (Z_0,Z_1,Z_2) = (0, 0, −2), (1, 0, 2), (1, 2, 2), each with probability 1/3. Call those events a, b, c. Then E(Z_1 | Z_0)(a) = 0 = Z_0(a); E(Z1|Z0)(b,c)=12(0+2)=Z0(b,c); E(Z2|Z1)(a,b)=12(−2+2)=Z1(a,b); E(Z_2 | Z_1)(c) = 2 = Z_1(c). But E(Z_2 | Z_0, Z_1)(a) = −2 ≠ Z_1(a).

(b) Yes. We have Σzn+1 (zn+1 − zn) Pr(Z_0 = z_0,...,Zn+1 = zn+1) = 0 for all fixed (z_0,...,zn). Sum these to get Σzn+1 (zn+1 − zn)Pr(Zn = zn,Zn+1 = zn+1) = 0.

65. Observe first that E(Zn+1 | Z_0,...,Zk) = E(E(Zn+1 | Z_0,...,Zn) | Z_0,...,Zk) = E(Zn | Z_0,...,Zk) whenever k < n. Thus E(Zm(n+1) | Z_0,...,Zm(n)) = Zm(n) for all n ≥ 0. Hence E(Zm(n+1) | Zm(0),...,Zm(n)) = Zm(n), as in the previous exercise.

66. We need to specify the joint distribution of {Z_0,...,Zn}, and it’s not difficult to see that there is only one solution. Let p(σ_1,...,σn) = Pr(Z_1 = σ_1,...,Zn = σnn) when σ_1, ... , σn are each ±1. The martingale law p(σ1...σn1)(n+1)−p(σ1...σn1¯)(n+1)=σnp(σ1...σn)n=σn(p(σ1...σn1)+p(σ1...σn1¯))n gives p(σ_1 ...σn+1)/p(σ_1 ...σn) = (1 + 2n[σnσn+1 > 0])/(2n + 2). Hence we find that Pr(Z1=z1,...,Zn=zn)=(Πk=1n−1(1+2k[zkzk+1>0]))/(2nn!). When n = 3, for example, the eight possible cases z1z2z3=123,123¯,...,1¯2¯3¯ occur with probabilities (15, 3, 1, 5, 5, 1, 3, 15)/48.

67. (a) You “always” (with probability 1) make 2n+1 − (1 + 2 + ··· +2n) = 1 dollar.

(b) Your total payments are X = X_0 + X_1 + ··· dollars, where Xn = 2n with probability 2−n, otherwise Xn = 0. So E Xn = 1, and E X = E X_0 + E X_1 + ··· = ∞.

(c) Let 〈Tn〉 be a sequence of uniformly random bits; and define the fair sequence Yn=(−1)Tn2nT0...Tn−1, or Yn = 0 if there is no nth bet. Then Zn = Y_0 + ··· + Yn.

[The famous adventurer Casanova lost a fortune in 1754 using this strategy, which he called “the martingale” in his autobiography Histoire de ma vie. A similar betting scheme had been proposed by Nicolas Bernoulli (see P. R. de Montmort, Essay d’Analyse sur les Jeux de Hazard, second edition (1713), page 402); and the perplexities of (a) and (b) were studied by his cousin Daniel Bernoulli, whose important paper in Commentarii Academiæ Scientiarum Imperialis Petropolitanæ 5 (1731), 175–192, has caused this scenario to become known as the St. Petersburg paradox.]

68. (a) Now Zn = Y_1 + ··· + Yn, where Yn=(−1)Tn[N≥n]. Again Pr(ZN = 1) = 1.

(b) The generating function g(z) equals z(1 + g(z)2)/2, since he must win $2 if the first bet loses. Hence g(z)=(1−1−z2)/z; and the desired probability is [zn] g(z) = C(n−1)/2[n odd]/2n, where Ck is the Catalan number (2kk)/(k+1).

(c) Pr(N≥n)=[zn](1−zg(z))/(1−z)=[zn](1+z)/1−z2=(2⌊n/2⌋⌊n/2⌋)/2⌊n/2⌋.

(d) E N = g′(1) = ∞. (It’s also Σn=1∞Pr(N≥n), where Pr(N≥n)∼1/πn.)

(e) Let pm = Pr(Zn ≥ −m) for all n ≥ 0. Clearly p_0 = 1/2 and pm = (1 + pm−1pm)/2 for m > 0; this recurrence has the solution pm = (m + 1)/(m + 2). So the answer is 1/((m + 1)(m + 2)); it’s another probability distribution with infinite mean.

(f) The generating function gm(z) for the number of times −m is hit satisfies g_0(z) = z/(2−z), gm(z) = (1+gm−1(z)gm(z))/2 for m > 0. So gm(z) = hm(z)/hm+1(z) for m ≥ 0, where hm(z) = 2m − (2m − 1)z, and gm′(1)=2. [A distribution with finite mean! See W. Feller, An Intro. to Probability Theory 2, second edition (1971), XII.2.]

69. Each permutation of n elements corresponds to a configuration of n + 1 balls in the urn. For Method 1, the number of corresponding “red balls” is the position of element 1; for Method 2, it is the value in position 1. For example, we’d put 3 1 2 4 into node (2, 3) with respect to Method 1 but into (3, 2) with respect to Method 2. (In fact, Methods 1 and 2 construct permutations that are inverses of each other.)

70. Start with the permutation 1 2 ... (c − 1) at the root, and use Method 1 of the previous exercise to generate all n!/(c−1)! permutations in which these elements retain that order. A permutation with j in position Pj for 1 ≤ j < c stands for Pj − Pj−1 balls of color j, where P_0 = 0 and Pc = n + 1; for example, if c = 3, the permutation 3 1 4 2 would correspond to node (2, 2, 1). The resulting tuples (A_1,...,Ac)/(n + 1) then form a martingale for n = c, c +1, ... , uniformly distributed (for each n) among all (nc−1) compositions of n + 1 into c positive parts.

[We can also use this setup to deal with Pólya’s two-color model when there are r red balls and b black balls at the beginning: Imagine r + b colors, then identify the first r of them with red. This model was first studied by D. Blackwell and D. Kendall, J. Applied Probability 1 (1964), 284–296.]

71. If m = r′ − r and n = b′ − b we must move m times to the right and n times to the left; there are (m+nn) such paths. Every path occurs with the same probability, because the numerators of the fractions are r⋅(r+1)⋅...⋅(r′−1)⋅b⋅(b+1)⋅...⋅(b′−1)=rm¯bn¯ in some order, and the denominators are (r+b)⋅(r+b+1)⋅...⋅(r′+b′−1)=(r+b)m+n¯.

The answer, (m+nn)rm¯bn¯/(r+b)m+n¯, reduces to 1/(r′ + b′ − 1) when r = b = 1.

72. Since all paths to (r, b) have the same probability, this expected value is the same as E(X_1X_2 ... Xm), which is obviously 1/(m+1). (Thus the X’s are very highly correlated: This expected value would be 1/2m if they were independent. Notice that the probability of an event such as (X_2 = 1, X_5 = 0, X_6 = 1) is E(X_2(1 − X_5)X_6) = 1/3 − 1/4.)

[The far-reaching ramifications of such exchangeable random variables are surveyed in O. Kallenberg’s book Probabilistic Symmetries and Invariance Principles (2005).]

73. f(r,n)=r(n+1r)Σk(r−1k)(−1)kqn+1−r+k, where qk = ak/(k+1), by induction on r.

74. Node (r, n + 2 − r) on level n is reached with probability 〈nr−1〉/n!, proportional to an Eulerian number (see Section 5.1.3). (Indeed, we can associate the permutations of {1, ..., n+1} that have exactly r runs with this node, using Method 1 as in exercise 69.)

Reference: Communications on Pure and Applied Mathematics 2 (1949), 59–70.

75. As before, let Rn = X_0+···+Xn be the number of red balls at level n. Now we have E(Xn+1 | X_0,...,Xn) = 1 − Rn/(n + 2). Hence E(Rn+1 | Rn) = (n +1)Rn/(n + 2) + 1, and the definition Zn = (n +1)Rn − (n + 2)(n +1)/2 is a natural choice.

76. No. For example, let Z_0 = X, Z0′=Y, and Z1=Z1′=X+Y, where X and Y are independent with E X = E Y = 0. Then E(Z_1 | Z_0) = Z_0 and E(Z1′|Z0′)=Z0′, but E(Z1+Z1′|Z0Z0′)=2(Z0+Z0′). (On the other hand, if 〈Zn〉 and 〈Zn′〉 are both martingales with respect to some common sequence 〈Xn〉, then 〈Zn+Zn′〉 is also.)

77. E(Zn+1 | Z_0,...,Zn) = E(E(Zn+1 | Z_0,...,Zn,X_0,...,Xn) | Z_0,...,Zn), which equals E(E(Zn+1 | X_0,...,Xn) | Z_0,...,Zn) because Zn is a function of X_0, ... , Xn; and that equals E(Zn | Z_0,...,Zn) = Zn. (Furthermore 〈Zn〉 is a martingale with respect to, say, a constant sequence. But not with respect to every sequence.)

A similar proof shows that any sequence 〈Yn〉 that is fair with respect to 〈Xn〉 is also fair with respect to itself.

78. E(Zn+1 | V_0,...,Vn) = E(ZnVn+1 | V_0,...,Vn) = Zn.

The converse holds with V_0 = Z_0 and Vn = Zn/Zn−1 for n > 0, provided that Zn−1 = 0 implies Zn = 0, and that we define Vn = 1 when that happens.

79. Zn = V_0V_1 ...Vn, where V_0 = 1 and each Vn for n > 0 is independently equal to q/p (with probability p) or to p/q (with probability q). Since E Vn = q + p = 1, 〈Vn〉 is multiplicatively fair. [See A. de Moivre, The Doctrine of Chances (1718), 102–154.]

80. (a) True; in fact E(fn(Y_0 ...Yn−1)Yn) = 0 for any function fn.

(b) False: For example, let Y_5 = ±1 if Y_3 > 0, otherwise Y_5 = 0. (Hence permutations of a fair sequence needn’t be fair. The statement is, however, true if the Y’s are independent with mean zero.)

(c) False if n_1 = 0 and m = 1 (or if m = 0); otherwise true. (Sequences that satisfy E((Yn_1 − E Yn_1) ... (Ynm − E Ynm)) = E(Yn_1 − E Yn_1) ... E(Ynm − E Ynm) are called totally uncorrelated. Such sequences, with E Yn = 0 for all n, are not always fair; but fair sequences are always totally uncorrelated.)

81. Assuming that X_0, ... , Xn can be deduced from Z_0, ... , Zn, we have anXn + bnXn−1 = Zn = E(Zn+1 | Z_0,...,Zn) = E(an+1Xn+1 + bn+1Xn | X_0,..., Xn) = an+1(Xn +Xn−1)+bn+1Xn for n ≥ 1. Hence an+1 = bn, bn+1 = an − an+1 = bn− 1 − bn; and we have an = F−n−1, bn = F−n−2 by induction, verifying the assumption.

[See J. B. MacQueen, Annals of Probability 1 (1973), 263–271.]

82. (a) Zn = An/Cn, where An = 4 − X_1 − ··· − Xn is the number of aces and Cn is the number of cards remaining after you’ve seen n cards. Hence E Zn+1 = (An/Cn)(An−1)/(Cn−1)+(1−An/Cn)An/(Cn−1) = An/Cn. (In every generalization of Pólya’s urn for which the nth step adds kn balls of the chosen color, the ratio red/(red + black) is always a martingale, even when kn is negative, as long as enough balls of the chosen color remain. This exercise represents the case kn = −1.)

(b) This is the optional stopping principle in a bounded-time martingale.

(c) ZN = AN /CN is the probability that an ace will be next. [“Ace Now” is a variant of R. Connelly’s game “Say Red”; see Pallbearers Review 9 (1974), 702.]

83. Zn=Σk=1n(Xn−EXn) is a martingale, for which we can study the bounded stopping rules Nn′(x0,...,xn−1)=[n<m]⋅Nn(x0,...,xn−1) for any m. But Svante Janson suggests a direct computation, beginning with the formula SN=Σn=1∞Xn[N≥n] where N might be ∞: We have E(Xn[N ≥ n]) = (E Xn)(E[N ≥ n]), because [N ≥ n] is a function of {X_0,...,Xn−1}, hence independent of Xn. And since Xn ≥ 0, we have ESN=Σn=1∞E(Xn[N≥n])=Σn=1∞(EXn)E[N≥n]=Σn=1∞E((EXn)[N≥n])=EΣn=1∞(EXn)[N≥n] which is EΣn=1NEXn. (The equation might be ‘∞ = ∞’.)

[Wald’s original papers, in Annals of Mathematical Statistics 15 (1944), 283–296, 16 (1945), 287–293, solved a somewhat different problem and proved more.]

84. (a) We have f(Zn) = f(E(Zn+1 | Z_0,...,Zn)) ≤ E(f(Zn+1) | Z_0,...,Zn) by Jensen’s inequality. And the latter is E(f(Zn+1) | f(Z_0),...,f(Zn)) as in answer 77. [Incidentally, D. Gilat has shown that every nonnegative submartingale is 〈|Zn|〉 for some martingale 〈Zn〉; see Annals of Probability 5 (1977), 475–481.]

(b) Again we get a submartingale, provided that we also have f(x) ≤ f(y) for a ≤ x ≤ y ≤ b. [J. L. Doob, Stochastic Processes (1953), 295–296.]

85. Since 〈Bn/(Rn + Bn) = 1 − Rn/(Rn + Bn)〉 is a martingale by (27), and since f(x) = 1/x is convex for positive x, 〈(Rn + Bn)/Bn〉 = 〈Rn/Bn + 1〉 is a submartingale by exercise 84. (A direct proof could also be given.)

86. The rule Nn+1(Z_0,...,Zn) = [max(Z_0,...,Zn) < x and n + 1 < m] is bounded. If max(Z_0,...,Zm−1) < x then we have ZN < x, where N is defined by (31); similarly, if max(Z_0,...,Zm−1) ≥ x then ZN ≥ x. Hence Pr(max(Z_0,...,Zn) ≥ x) ≤ (E ZN )/x by Markov’s inequality; and E ZN ≤ E Zn in a submartingale.

87. This is the probability that Zn becomes 3/4, which also is Pr(max(Z_0,...,Zn) ≥ 3/4). But E Zn = 1/2 for all n, hence (33) tells us that it is at most (1/2)/(3/4) = 2/3.

(The exact value can be calculated as in the following exercise. It turns out to be Σk=0∞2(4k+2)(4k+3)=12H3/4−12H1/2+13=14π−12ln 2≈.439.)

88. (a) We have S > 1/2 if and only if there comes a time when there are more red balls than black balls. Since that happens if and only if the process passes through one of the nodes (2, 1), (3, 2), (4, 3), ... , the desired probability is p_1 + p_2 + ··· , where pk is the probability that node (k + 1, k) is hit before any of (j + 1, j) for j < k.

All paths from the root to (k + 1, k) are equally likely, and the paths that meet our restrictions are equivalent to the paths in 7.2.1.6–(28). Thus we can use Eq. 7.2.1.6–(23) to show that pk = 1/(2k − 1) − 1/(2k); and 1 − 1/2 + 1/3 − 1/4 + ··· = ln 2.

(b, c) If pk is the probability of hitting node ((t − 1)k +1, k) before any previous ((t − 1)j +1, j), a similar calculation using the t-ary ballot numbers Cpq(t) yields pk = (t − 1)(1/(tk − 1) − 1/(tk)). Then Σk=1∞pk=1−(1−1/t)H1−1/t (see Appendix A).

Notes: We have Pr(S = 1/2) = 1 − ln 2, since S is always ≥ 1/2. But we cannot claim that Pr(S ≥ 2/3) is the sum of cases that pass through (2, 1), (4, 2), (6, 3), etc., because the supremum might be 2/3 even though the value 2/3 is never reached. Those cases occur with probability π/27 hence Pr(S=2/3)≥2π/27−ln3≈.111. A determination of the exact value of Pr(S = 2/3) is beyond the scope of this book, because we’ve avoided the complications of measure theory by defining probability only in discrete spaces; we can’t consider a limiting quantity such as S to be a random variable, by our definitions! But we can assign a probability to the event that max(Z_0,Z_1,...,Zn) > x, for any given n and x, and we can reason about the limits of such probabilities.

With the help of deeper methods, E. Schulte-Geers and W. Stadje have proved that the supremum is reached within n steps, a.s. Hence Pr(S=2/3)=2π/27−ln3; indeed, Pr(S is rational) = 1, since only rationals are reached; and Pr(S = (t − 1)/t) = (2−3/t)H1−1/t−(1−2/t)H1−2/t−(t−2)/(t−1). [J. Applied Prob. 52 (2015), 180–190.]

89. Set Yn = cn(Xn − pn), an = −cnpn, bn = cn(1 − pn). (Incidentally, when c_1 = ··· = cn = 1, exercise 1.2.10–22 gives an upper bound that has quite a different form.)

90. (a) Apply Markov’s inequality to Pr(e(Y1+...+Yn)t≥etx).

(b) eyt ≤ e−pt(q − y) + eqt(y + p) = ef(t) + yeg(t) because the function eyt is convex.

(c) We have f′(t) = −p + pet/(q + pet) and f″(t) = pqet/(q + pet)2; hence f(0) = f′(0) = 0. And f″(t) ≤ 1/4, because the geometric mean of q and pet, (pqet)1/2, is less than or equal to the arithmetic mean, (q + pet)/2.

(d) Set c = b − a, p = −a/c, q = b/c, Y = Y/c, t = ct, h(t) = eg(ct)/c.

(e) In E((ec12t2/4+Y1h1(t))...(ecn2t2/4+Ynhn(t))) the terms involving hk(t) all drop out, because 〈Yn〉 is fair. So we’re left with the constant term, ect2/4.

(f) Let t = 2x/c, to make ct^2/4 − xt = −x^2/c.

91. E(Zn+1 | X_0,...,Xn) = E (E(Q | X_0,...,Xn,Xn+1) | X_0,...,Xn), and this is equal to E(Q | X_0,...,Xn) by formula (∗) in answer 35. Apply exercise 77.

92. Q_0 = E Xm = 1/2. If n < m we have Qn = E(Xm | X_0,...,Xn), which is the same as E(Xn+1 | X_0,...,Xn) (see exercise 72); and this is (1 + X_1 + ··· + Xn)/(n + 2), which is the same as Zn in (27). If n ≥ m, however, we have Qn = Xm.

93. Everything goes through exactly as before, except that we must replace the quantity (m − 1)t/mt−1 by the generalized expected value, which is Σk=1mΠn=1t(1−pnk).

94. If the X’s are dependent, the Doob martingale still is well defined; but when we write its fair sequence as an average of Δ(x_1,...,xt) there is no longer a nice formula such as (40). In any formula for Δ that has the form Σxpx(Q( ...xn... ) − Q( ... x ... )), Pr(Xn = xn, Xn+1, ... )/(Pr(Xn = xn) Pr(Xn+1 = xn+1, ... )) must equal Σx px, so it must be independent of xn. Thus (41) can’t be used.

95. False; the probability of only one red ball at level n is 1/(n+1) = Ω(n−1). But there are a.s. more than 100 red balls, because that happens with probability (n−99)/(n+1).

96. Exercise 1.2.10–21, with ∊n equal to the bound on |X − n/2|, tells us that (i) is q.s. and that (i), (ii), (iii) are a.s. To prove that (iv) isn’t a.s., we can use Stirling’s approximation to show that (nn/2±k)/2n is ⊖(n−1/2 when k=n; consequently Pr(|X|<n)=⊖(1). A similar calculation shows that (ii) isn’t q.s.

97. We need to show only that a single bin q.s. receives that many. The probability generating function for the number of items H that appear in any particular bin is G(z) = ((n − 1 + z)/n)N, where N = ⌊n1+δ⌋ . If r=12nδ, we have

Pr(H≤r)≤(12)−rG(12)=2r(1−12n)⌊2nr⌋≤2r(1−12n)2nr−1≤2r+1e−r,
by 1.2.10–(24). And if r = 2nδ we have

Pr(H≥r)≤2−rG(2)=2−r(1+1n)⌊nr/2⌋≤2−r(1+1n)nr/2≤2−rer/2,
by 1.2.10–(25). Both are exponentially small. [See Knuth, Motwani, and Pittel, Random Structures & Algorithms 1 (1990), 1–14, Lemma 1.]

98. Let En = E R, where R is the number of reduction steps; and suppose F(n) = k with probability pk, where Σk=1npk=1 and Σk=1nkpk=g≥gn. (The values of p_1, ... , pn, and g might be different, in general, every time we compute F (n).)

Let Σab=Σj=ab1/gj. Clearly E_0 = 0. And if n > 0, we have by induction

En=1+Σk=1npkEn−k≤1+Σk=1npkΣ1n−k=1+Σk=1npk(Σ1n−Σ1n−Σn−k+1n)=Σ1n+1−Σk=1npkΣn−k+1n≤Σ1n+1−Σk=1npkkgn≤Σ1n.

[See R. M. Karp, E. Upfal, and A. Wigderson, J. Comp. and Syst. Sci. 36 (1988), 252.]

99. The same proof would work, provided that induction could be justified, if we were to do the sums from k = −∞ to n and define Σab=−Σj=b+1a−11/gj when a > b. (For example, that definition gives −Σn+3n=1/gn+1+1/gn+2≤2/gn.)

And in fact it does become a proof, by induction on m, that we have Em,n≤Σ1n for all m, n ≥ 0, where Em,n = E min(m, R). Indeed, we have E_0,n = Em+1,0 = 0; and Em+1,n=1+Σk=−∞npkEm,n−k when n > 0. [This problem is exercise 1.6 in Randomized Algorithms by Motwani and Raghavan (1995). Svante Janson observes that the random variable Zm=Σ1Xm+min(m,R) is a supermartingale, where Xm is the value of X after m iterations, as a consequence of this proof.]

100. (a) Σk=1mkpk≤Emin(m,T)=p1+2p2+⋅⋅⋅+mpm+mpm+1+⋅⋅⋅+mp∞≤ET.

(b) E min(m, T) ≥ mp∞ for all m. (We assume that ∞· p = (p > 0? ∞: 0).

101. (Solution by Svante Janson.) If 0 < t < min(p_1,...,pm) = p, we have EetX=∏k=1mEetXk=∏k=1mpk/(e−t−1+pk)<∏k=1mpk/(pk−t), because e−t−1 > −t. Set t = θ/μ, and note that pkln(1−t/pk)≥pln(1−t/p)≥1μln(1−tμ)=tθln(1−θ). By 1.2.10– (25), therefore, Pr(X≥rμ)≤e−rtμ∏k=1mpk/(pk−t)=exp(−rθ−Σk=1mln(1−t/pk))≤exp(−rθ−Σk=1m(t/pk)(ln(1−θ))/θ)=exp(−rθ−ln(1−θ)). Choose θ = (r − 1)/r to get the desired bound re1−r. (The bound is nearly sharp when m = 1 and p is small, since Pr(X ≥ r/p) = (1 − p)⌈r/p⌉−1 ≈ e−r.)

102. Applying exercise 101 with μ ≤ s_1 + ··· + sm and r = ln n gives probability O(n−1 log n) that (s_1 + ··· + sm)r trials aren’t enough. And if r = f(n)ln n, where f(n) is any increasing function that is unbounded as n → ∞, the probability that skr trials don’t obtain coupon k is superpolynomially small. So is the probability that any one of a polynomial number of such failures will occur.

103. (a) The recurrence poij = [i=j], p(n+1)ij=Σk=02pnik([f0(k)=j]+[f1(k)=j])/2 leads to generating functions gij=Σn=0∞pnijzn that satisfy gi0 = [i=0]+(gi0+gi1)z/2, gi1 = [i=1]+(gi0+gi2)z/2, gi2 = [i=2]+(gi1+gi2)z/2. From the solution gi_0 = A+B+C, gi_1 = A−2B, gi_2 = A+B−C, A=13/(1−z), B=16(1−3[i=1])(1+z/2), and C=12([i=0]−[i=2])/(1−z/2), we conclude that the probability is 13+O(2−n); in fact it is always either ⌊2n/3⌋/2n or ⌈2n/3⌉/2n. The former occurs if and only if i ≠ j and n is even, or i + j = 2 and n is odd.

(b) Letting g012=z2(g001+g112), g001=z2([j=0]+g011), etc., yields the generating function g_012 = ([j ≠ 1] + [j = 1]z)z^2/(4 − z^2). Hence each j occurs with probability 1/3, and the generating function for N is z^2/(2−z); mean = 3, variance = 2.

(c) Now g001=z2([j=0]+g012), etc.; the output is never 1; 0 and 2 are equally likely; and N has the same distribution as before.

(d) Functional composition isn’t commutative, so the stopping criterion is different: In the second case, 111 cannot occur unless the previous step had 000 or 222. The crucial difference is that, without stopping, process (b) becomes fixed at coalescence; process (c) continues to change a_0a_1a_2 as n increases (although all three remain equal).

(e) If T is even, sub(T ) returns (−1, 0, 1, 2) with probability (2, (2T − 1)/3, (2T − 4)/3, (2T − 1)/3)/2T . Thus the supposed alternative to (b) will output 0 with probability 14+532+854096+⋅⋅⋅=13Σk=1∞2k+1(22k−1)/22k+1≈0.427, not 1/3.

(f) Change sub(T) to use consistent bits XT , XT−1, ... , X_1 instead of generating new random bits X each time; then the method of (b) is faithfully simulated. (The necessary consistency can be achieved by carefully resetting the seed of a suitable random number generator at appropriate times.)

[The technique of (f) is called “coupling from the past” in a monotone Monte Carlo simulation. It can be used to generate uniformly random objects of many important kinds, and it runs substantially faster than method (b) when there are thousands or millions of possible states instead of just three. See J. G. Propp and D. B. Wilson, Random Structures & Algorithms 9 (1996), 223–252.]

104. Let q = 1 − p. The probability of output (0, 1, 2) in (b) is (q^2, 2pq, p^2); in (c) it is (p^2+pq^2, 0, q^2+qp^2). In both cases N has generating function (1−pq(2−z))z^2/(1−pqz^2), mean 3/(1 − pq) − 1, variance (5 − 2pq)pq/(1 − pq)2.

105. We have g_0 = 1 and ga = z(ga−1 + ga+1)/2 for 0 < a < n/2.

If n = 2m is even, let ga = zatm−a/tm for 0 ≤ a ≤ m. The polynomials tk defined by t_0 = t_1 = 1, tk+1 = 2tk − z^2 tk−1 fill the bill, because they make gm = zgm−1. The generating function Tw = ∑m = 0∞ tmwm = 1 - w/1 - 2w + w2 z2 now shows, after differentiation by z, that we have tm'1 = -mm - 1 and tm''(1) = (m2 - 5m + 3)m(m-1)/3; hence tm''1 +tm'1 - tm'12 = 23m2 - m4. The mean and variance, given a, are therefore a − (m − a)(m − a − 1) + m(m − 1) = a(n − a) and 23m-a2 - m-a4 - m2 + m4 = 13n-a2 + a2 - 2an-a, respectively.

When n = 2m − 1 we can write ga = za um−a/um for 0 ≤ a ≤ m, with um+1 = 2 um − z^2 um − 1. In this case we want u_0 = 1 and u_1 = z, so that gm = gm−1. From Uw = ∑m = 0∞ umwm = 1 + z - 2w/1-2w + w2z2 we deduce um'1 = -mm-2 and um''1 = mm-1m2 - 7m + 7/3. It follows that, also in this case, the mean number of steps in the walk is a(n − a) and the variance is 13n-a2 + a2 -2an-a.

[The polynomials tm and um in this analysis are disguised relatives of the classical Chebyshev polynomials defined by Tmcos θ = cos mθ, Umcos θ = sinm +1θ/sin θ. Let us also write Vmcos θ = cosm - 12θ/cos 12θ. Then Vm(x) = (2 − 1/x) Tm(x)+ (1/x − 1) Um(x); and we have tm = zmTm(1 /z), um = zmVm(1/z).]

106. Before coalescing, the array a_0 a_1 ... ad−1 always has the form ar(a+1) ... (b−1) bs for some 0 ≤ a < b < d, r > 0, and s > 0, where r + s + b − a = d + 1. Initially a = 0, b = d − 1, r = s = 1. The behavior of the algorithm while r+ s = t is like a random walk on the t-cycle, as in the previous exercise, starting at a = 1. Let Gt be the generating function for that problem, which has mean t − 1 and variance 23t. Then this problem has the generating function G_2 G_3 ... Gd; so its mean is ∑k = 2d k - 1 = 2d, and the variance is ∑k=2d 23k = 24d+1.

107. (a) If the probabilities can be renumbered so that p_1 ≤ q_1 and p_2 ≤ q_2, the five events of Ω can have probabilities p_1, p_2, q_1 − p_1, q_2 − p_2, and q_3, because p_3 = (q_1 − p_1)+ (q_2 − p_2)+ q_3. But if that doesn’t work, we can suppose that p_1 < q_1 ≤ q_2 ≤ q_3 < p_2 ≤ p_3. Then p_1, q_1 − p_1, p_1 + p_2 − q_1, p_3 − q_3, and q_3 are nonnegative.

(b) Give Ω’s events the probabilities 112,212,312,612.

(c) For example, let p1 = 19, p2 = p3 = 49, q1 = q2 =q3 = 13.

108. Let pk = Pr′(X = k) and qk = Pr″(Y = k). The set ∪n∑k≤n pk, ∑k≤n qk divides the unit interval [0 .. 1) into countably many subintervals, which we take as the set Ω of atomic events ω. Let X(ω) = n if and only if ω ⊆[∑k<n pk .. ∑k≤n pk); a similar definition works for Y (ω). And X(ω) ≤ Y (ω) for all ω.

109. (a) We’re given that p_1 + p_3 ≤ q_1 + q_3, p_2 + p_3 ≤ q_2 + q_3, and p_3 ≤ q_3. (Also that 0 ≤ 0 and p_1 + p_2 + p_3 ≤ q_1 + q_2 + q_3; but those inequalities always hold.) We must find a coupling with p_12 = p_21 = p_31 = p_32 = 0, because 1⊀2,2⊀1,3⊀1, and 3⊀2. In the previous problem we were given that p_2 + p_3 ≤ q_2 + q_3 and p_3 ≤ q_3, and we had to find a coupling with p_21 = p_31 = p_32 = 0.

(b) Let A↑ = { x | x ⪰ a for some a ∈ A} and B↓ = { x | x ⪯ b for some b ∈ B}. We’re given that Pr′(X ∈ A↑) ≤ Pr″(Y ∈ A↑) for all A. Let A = {1 ,..., n}\ B↓, so that Pr′(X ∈ B↓) = 1 − Pr′(X ∈ A). The result follows because A = A↑.

(c) Remove all arcs xi → xj from the network when i⊀j. Then a blocking pair (I, J) has the property that i j implies i ∈ I or j ∈ J. Let A = { x | x ⪯ a for some a ∉ J} and B = {1 ,..., n}\ A. Then A ⊆ I, B ⊆ J, and B = B↓. Hence Σi∈ I pi + Σj∈J qj ≥ Σi∈A pi + Σj∈B qj ≥ Σi∈A qi + Σj∈B qj = 1.

[See K. Nawrotzki, Mathematische Nachrichten 24 (1962), 193–200; V. Strassen, Annals of Mathematical Statistics 36 (1965), 423–439.]

110. (a) The result is trivial if r = 1. Otherwise consider the probability distributions pk' = pk rk/1 - r and qk' = qk rk/1 - r; use the coupling pij = 1-r pi' qj' + rj i = j. [See W. Doeblin, Revue mathématique de l’Union Interbalkanique 2 (1938), 77–105; R. L. Dobrushin, Teoriya Veroyatnosteĭ i ee 15 (1970), 469–497.]

(b) Yes, because the (p′, q′) distribution satisfies the hypotheses of that exercise.

111. (a) Here are the 60 triples 1π 3π 4π, with the minima in bold type:

134 163 123 126 142 142 153 145 163 154 245 234 534 563 623 526 632 652 534 643
356 645 246 234 435 463 524 423 642 532 461 351 361 641 251 231 341 531 321 421
512 412 415 315 316 615 216 216 415 316 623 526 652 452 564 354 465 364 256 265

(b) Both SA and SB lie in A ∪ B. Each element of A ∪ B is equally likely to have the minimum value aπ; exactly | A ∩ B| of those elements have that value as their sketch.

(c) | A ∩ B ∩ C|/| A ∪ B ∪ C|.

Notes: The ratio | A ∩ B|/| A∪ B| is a useful measure of similarity, called the “Jaccard index” because Paul Jaccard used it to compare different Swiss ecological sites according to the sets of plant species seen at each place [Bulletin de la Société Vaudoise des Sciences Naturelles 37 (1901), 249]. It is commonly used today to rank the similarity between web pages, based on a certain set of words in each page.

Minwise independence was introduced by Andrei Broder for that application in 1997, using n = 2^64 and a method of identifying roughly 1000 words A on a typical web page. By calculating, say, independent sketches S_1(A), ... , S_100(A) for each page, the number of j such that Sj(A) = Sj(B) gives a highly reliable and quickly computable estimate of the Jaccard index. A perfectly minwise independent family is impossible in practice when n is huge, but the associated theory has led to approximate “minhash” algorithms that work well. See A. Z. Broder, M. Charikar, A. M. Frieze, and M. Mitzenmacher, J. Computer and System Sciences 60 (2000), 630–659. See also the related, independent work by K. Mulmuley, Algorithmica 16 (1996), 450–463.

112. (a) Such a rule breaks ties properly, provided that the number of π with ∞’s in B is a multiple of n − m. Each B can have its own rule.

(b) In fact we can produce families whose permutations are all obtained from N/n = d “seeds” by cyclic shifts, as in exercise 111. Begin with m = 1 and a table of N = lcm(1, 2, ..., n) partial permutations whose entries πij for 1 ≤ i ≤ N and 1 ≤ j ≤ n are entirely blank, except that πij = 1 for each pair ij with (j − 1) d < i ≤ jd and 1 ≤ j ≤ n. When n = 4, for instance, the initial tableau

1⊔⊔⊔   1⊔⊔⊔   1⊔⊔⊔   ⊔1⊔⊔   ⊔1⊔⊔   ⊔1⊔⊔   ⊔⊔1⊔   ⊔⊔1⊔   ⊔⊔1⊔   ⊔⊔⊔1   ⊔⊔⊔1   ⊔⊔⊔1
represents N = 12 truncated permutations with m = 1. We’ll insert some 2s next.

Let A be a subset of size n − m that is all blank, in some π. Each A occurs equally often (as in uniform probing, Section 6.4); so the number of such π is N/n-mn. Fortunately this is a multiple of n − m, because exercise 1.2.6–48 tells us that N/n-mn-mn = N∑k=0m-1kkm/n-m+k.

Take n− m such π and insert m+1 into different positions within them. Then find another such A, if possible, and repeat the process until no blank subsets of size n − m remain. Then set m ← m + 1, and continue in the same way until m = n.

It’s not hard to see that the insertions can be done so that πj, πd+j, ... , π(n−1)d+j are maintained as cyclic shifts of each other. When n = 4 the 2s are essentially forced:

12⊔⊔   1⊔2⊔   1⊔⊔2   ⊔12⊔   ⊔1⊔2   21⊔⊔   ⊔⊔12   2⊔1⊔   ⊔21⊔   2⊔⊔1   ⊔2⊔1   ⊔⊔21

But then there are two ways to fill the two cases with A = {3 , 4}:

123⊔   1⊔2⊔   13⊔2   ⊔123   ⊔1⊔2   21⊔3   3⊔12   2⊔1⊔   ⊔213   23⊔1   ⊔2⊔1   3⊔21
12⊔3   1⊔2⊔   13⊔2   312⊔   ⊔1⊔2   213⊔   ⊔312   2⊔1⊔   ⊔213   2⊔31   ⊔2⊔1   3⊔21

Adopting the first of these leads to two ways to fill A = {2, 4}:

123⊔   132⊔   13⊔2   ⊔123   ⊔132   21⊔3   3⊔12   2⊔13   ⊔213   23⊔1   32⊔1   3⊔21
123⊔   1⊔23   13⊔2   ⊔123   31⊔2   21⊔3   3⊔12   231⊔   ⊔213   23⊔1   ⊔231   3⊔21

Here A is a cyclic shift of itself, but consistent placement is always possible.

[See Yoshinori Takei, Toshiya Itoh, and Takahiro Shinozaki, IEICE Transactions on Fundamentals E83-A (2000), 646–655, 747–755.]

113. (a) The probability is zero if l ≥ k or r > n − k. Otherwise the result follows if we can prove it in the “complete” case when l = k − 1 and r = n − k, because we can sum the probabilities of complete cases over all ways to specify which of the unconstrained elements are < k and which are > k.

To prove the complete case, we may assume that ai = i, b = k, and cj = k + j for 1 ≤ i ≤ l = k − 1 and 1 ≤ j ≤ r = n − k. The probability can be computed via the principle of inclusion and exclusion, because we know Pr(mina∈A aπ = kπ) = 1/(n − k + t) = PB whenever A = { k,..., n} ∪ B and B consists of t elements less than k. For example, if k = 4 the probability that 4 π = 4 and {1 π, 2 π, 3 π} = {1 , 2 , 3} is P∅ − P{1} − P{2} − P{3} + P{1, 2} + P{1, 3} + P{2,3} − P{1,2,3}; each of those probabilities is correct for truly random π.

(b) This event is the disjoint union of complete events of type (a). [See A. Z. Broder and M. Mitzenmacher, Random Structures & Algorithms 18 (2001), 18–30.]

Notes: The function ψ(n) = ln(lcm(1, 2, ..., n)) = Σpk≤n[p prime] ln p was introduced by P. L. Chebyshev [see J. de mathématiques pures et appliquées 17 (1852), 366–390], who proved that it is Θ(n). Refinements by Ch.-J. de la Vallée Poussin [Annales de la Société Scientifique de Bruxelles 20 (1896), 183–256] showed that in fact ψ(n) = n + O(ne−C log n) for some positive constant C. Thus lcm(1, 2, ..., n) grows roughly as en, and we cannot hope to generate a list of minwise independent permutations when n is large; the length of such a list is 232,792,560 already for 19 ≤ n ≤ 22.

114. First assume that | Sj| = dj + 1 for all j, and let gj(x) = ∏s∈Sj (x − s). We can replace xjdj + 1 by xjdj + 1 - gj xj, without changing the value of f(x_1 ,..., xn) when xj ∈ Sj. Doing this repeatedly until every term of f has degree ≤ dj in each variable xj will produce a polynomial that has at least one nonroot in S_1 × … × Sn, according to exercise 4.6.1–16. [See N. Alon, Combinatorics, Probab. and Comput. 8 (1999), 7–29.]

Now in general, if there were at most | S_1|+…+| Sn|−(d_1 +…+ dn + n) nonroots, we could eliminate them one (or more) at a time, by removing an element from any Sj for which | Sj| ≥ dj + 1. Contradiction.

(This inequality also implies stronger lower bounds when the sets Sj are large. If, for example, d_1 = … = dn = d and if each | Sj| ≥ s, where s = d + 1 + ⌈d/(n − 1)⌉, we can decrease each | Sj| to s and increase the right-hand side. For further asymptotic improvements see Béla Bollobás, Extremal Graph Theory (1978), §6.2 and §6.3.)

115. Representing the vertex in row x and column y by (x, y), if all points could be covered we’d have fx, y = ∏j = 1p x - aj∏j=1qy- bj∏j=1rx+y+cjx-y+dj = 0, for all 1 ≤ x ≤ m and 1 ≤ y ≤ n and for some choices of aj, bj, cj, dj. But f has degree p + q +2 r = m + n − 2, and the coefficient of xm-1yn-1 is ±r/2r≠0.

116. Let gv = Σ{xe | v ∈ e} for each vertex v, including xe twice if e is a loop from υ to itself. Apply the nullstellensatz with f = ∏v 1 - gvp-1-∏e1 - xe and with each Sj = {0, 1}, using mod p arithmetic. This polynomial has degree m, the number of edges and variables, because the first product has degree (p − 1) n < m; and the coefficient of ∏e xe is (−1)m ≠ 0. Hence there is a solution x that makes f(x) nonzero. The subgraph consisting of all edges with xe = 1 in this solution is nonempty and satisfies the desired condition, because gv(x) mod p = 0 for all υ.

(This proof works also if we consider that a loop contributes just 1 to the degree. See N. Alon, S. Friedland, and G. Kalai, J. Combinatorial Theory B37 (1984), 79–91.)

117. If ω = e2πi/m, we have EωjX = ∑k=0n knpk(1 -p)n-kωjk = (ωj p +1 -p)n. Also |ωj p + 1 − p|2 = p^2 + 1−p^2 + p(1−p)(ωj + ω−j = 1 −4p(1−p sin^2(πj/m). Now sin πt ≥ 2t for 0 ≤ t ≤ 1/2. Hence, if 0 ≤ j ≤ m/2 we have | ωj p + 1 − p|2 ≤ 1 − 16 p(1 − p) j^2/m^2 ≤ exp(−16 p(1 − p) j^2/m^2); if m/2 ≤ j ≤ m we have sin(πj/m) = sin(π(m − j)/m). Thus ∑j=1m-1|EωjX|≤2 ∑j=1m-1 exp-8p1-pj2n/m2.

The result follows, since PrX mod m=r = 1m∑j=0m-1ω-jr EωjX. [S. Janson and D. E. Knuth, Random Structures & Algorithms 10 (1997), 130–131.]

118. Indeed, (22) with Y = X − x yields more (when we also apply exercise 47):

Pr(X≥x)≥Pr(X>x)≥(E  X−x)2E ( X−x)2=(EX−x)2E X2−x(2⁢ E X−x)≥(E X−x)2E X2−x E X≥(EX−x)2E (X2−x)2.

(The attribution of this result to Paley and Zygmund is somewhat dubious. They did, however, write an important series of papers [Proc. Cambridge Philosophical Society 26 (1930), 337–357, 458–474; 28 (1932), 190–205] in which a related inequality appeared in the proof of Lemma 19.)

119. Let f(x, t) = Pr(U ≤ V ≤ W and V ≤ (1 − t) U + tW ), g(x, t) = Pr(U ≤ W ≤ V and W ≤ (1 − t) U + tV ), h(x, t) = Pr(W ≤ U ≤ V and U ≤ (1 − t) W + tV ). We want to prove that f(x, t) + g(x, t) + h(x, t) = t. Notice that, if U¯=1−U, V¯=1−V, W¯=1−W, we have Pr(W ≤ U ≤ V and U≥(1−t)W+tV)=Pr(V≤U¯≤W¯ and U¯≤tV¯+1(1−t)W¯). Hence x2-h(x,t)=f(1-x, 1-t), and we may assume that t ≤ x.

Clearly g(x,t)= ∫0xdux∫x1dv1-xt(v-u)=t2. And t ≤ x implies that
f(x,t)=∫(x−t)/(1−t)x dux⁢∫x(1−t)u+t dv1−x(v−(1−t)u)/t)=t2(1−x)2/(6(1−t)x);h(x,t)=∫x1  dv1−x(∫0vt  duxu+∫vtx dux⁢t1−t(v−u))=t2−f(x,t).

Instead of this elaborate calculation, Tamás Terpai has found a much simpler proof: Let A = min(U, V, W ), M = UV W , and Z = max(U, V, W ). Then the conditional distribution of M, given A and Z, is a mixture of three distributions: Either A = U, Z = V , and M is uniform in [ A..Z]; or A = U, Z = W , and M is uniform in [ x..Z]; or A = W , Z = V , and M is uniform in [ A..x]. (These three cases occur with respective probabilities (Z − A, Z − x, x − A)/(2 Z − 2 A), but we don’t need to know that detail.) The overall distribution of M, being an average of conditional uniform distributions over all A ≤ x and Z ≥ x, is therefore uniform.

[See S. Volkov, Random Struct. & Algorithms 43 (2013), 115–130, Theorem 5.]

120. See J. Jabbour-Hattab, Random Structures & Algorithms 19 (2001), 112–127.

121. (a) D(y∥x)=15lg⁢ 65+215lg45≈.0097; D(x∥y)=16lg⁢ 56+16lg54≈.0098.

(b) We have E(ρ(X) lg ρ(X)) ≥ (Eρ(X)) lg Eρ(X) by Jensen’s inequality (20); and E ρ(X) = Σt y(t) = 1, so the logarithm evaluates to 0.

The question about zero is the hard part of this exercise. We need to observe that the function f(x) = x lg x is strictly convex, in the sense that equality holds in (19) only when x = y. Thus we have (E Z)lgE Z = E(Z lg Z) for a positive random variable Z only when Z is constant. Consequently D(y|| x) = 0 if and only if x(t) = y(t) for all t.

(c) Let x^⁢(t)=x(t)/p and y^⁢(t)=y(t)/q be the distributions of X and Y within T. Then 0≤D(y^∥x^)=∑t∈T y^(t)⁢lg(y^(t)/x^(t))=E(lg⁢ ρ⁢(Y)|Y∈T)+lg(p/q).

(d) D(y|| x) = (Elg m) − HY = lg m − HY. (Hence, by (b), the maximum entropy of any such random variable Y is lg m, attainable only with the uniform distribution. Intuitively, HY is the number of bits that we learn when Y is revealed.)

(e) IX,Y = − HZ −Σu,v z(u, v)(lg x(u)+lg y(v)) = − HZ + Σu x(u)lg(1/x(u)) + Σv y(v) lg(1/y(v)), because Σv z(u, v) = x(u) and Σu z(u, v) = y(v).

(f) Conditioning IX,Z = HX + HZ − HX,Z on Y gives 0 ≤ I(X,Z)|Y = HX|Y + HZ|Y − H(X,Z)|Y = HX|Y + (HY,Z − HY ) − (HX,Y,Z − HY).

122. (a) D(y∥x)=∑t=0∞(3t/4t+1)lg(3t/2t+1)=lg2716≈0.755;  D(x∥y)=lg43≈0.415.

(b) Let q = 1 − p and t=pn+un. Then we have
yt=e-u2/2pq2πpqnexpu2q-u2p+u36p2-u36q21n+O(1n);ln ρt=-u22q-12ln q +u2q-u36q21n+O1n.

By restricting | u| ≤ n∈ and trading tails (see 7.2.1.5–(20)), we obtain

Dy∥x=12πpqn∫-∞∞e-u2/2pq-u22q ln 2-12lg qdun+O1n=12 ln 2ln11-p-p+O1n.

In this case D(x|| y) is trivially ∞, because x(n +1) > 0 but y(n +1) = 0.

123. Since pk+1 = pk y(t)/zk(t) we have ρ(t) = (1 − pk) pk+1/(pk(1 − pk+1)). [This relation was the original motivation that led S. Kullback and R. A. Leibler to define D(y|| x), in Annals of Mathematical Statistics 22 (1951), 79–86.]

124. Let m = c22Dy∥x and g(t) = f(t)[ ρ(t) ≤ m]; thus g(t) = f(t) except with probability Δc. We have | E(f)− En(f)| = (E(f)− E(g))+ |E(g)− En(g)|+( En(f)− En(g)). The Cauchy–Schwarz inequality (exercise 1.2.3–30) implies that the first and last are bounded by ∥f∥∆c, because f(t) − g(t) = f(t)[ ρ(t) >m].

Now var(ρ(X) g(X)) ≤ E( ρ(X)2 g(X)2) ≤ m E(ρ(X) f(X)2) = m E(f(Y )2) = m f 2. Hence (E(g) − En(g))2 = var En(g) = var(ρ(X) g(X))/n ≤ ||f||2/c^2.

Consider now the case c < 1. From Markov’s inequality we have Pr(ρ(X) > m) ≤ (E ρ(X))/m = 1/m. Also E(ρ(X)[ ρ(X) ≤ m]) = E[ ρ(Y ) ≤ m] = 1 − Δc. Consequently Pr(En(1) ≥ a) ≤ Pr(max1≤k≤n ρ(Xk) > m) + Pr(∑k=1n  ρ(Xk)[ρ(Xk)≤m]≥na)≤n/m+E(∑k=1n ρ(Xk)[ρ(Xk)⁢≤ m])/(na)=c2+(1-∆c)/a.

[S. Chatterjee and P. Diaconis, Annals of Applied Prob. 28 (2018), 1099–1135.]

125. (a) From an2 = an-1an+1 we deduce that an = cxn for some c ≥ 0 and x ≥ 0.

(b) It remains log-convex ⇔ ca_1 ≥ a02; it remains log-concave ⇔ ca_1 ≤ a02. (The latter condition always holds in the important case c = 0.)

(c) If am−1 an+1 > 0 we have am/am−1 ≥ am + 1/am ≥ … ≥ an + 1/an, because there are no internal zeros. (And the analogous result holds for log-convexity.)

(d) If xz ≥ y^2 and XZ ≥ Y^2 and x, y, z, X, Y, Z > 0, we have (x + X)(z + Z) − (y + Y )2 ≥ (x + X)(y^2/x + Y^2/X) − (y + Y )2 = (x/X)(Y − Xy/x)2 ≥ 0. [L. L. Liu and Y. Wang, Advances in Applied Mathematics 39 (2007), 455.]

(e) Let cn=∑k(nk)akbn-k. Clearly c12 ≤ c0c2. And cn = ∑kkn-1an-1-kbk+1 + ∑kkn-1ak+1bn-1-k, so we can apply (c) and induction on n to the shifted sequences. [H. Davenport and G. Pólya, Canadian Journal of Mathematics 1 (1949), 2–3.]

(f) Yes: Let ak = bk = 0 when k < 0, and cn = Σk ak bn−k. Then we have
cn2-cn-1cn+1=∑0≤j≤k(ajak-aj-1ak+1)(bn-jbn-k-bn+1-jbn-1-k),
which is a special case of the Binet–Cauchy identity (exercise 1.2.3–46) with m = 2.

(g) Yes, but a more intricate proof seems to be needed. We have cn = t_00, cn+1 = t_01 + t_10, and cn+2 = t_02 + 2 t_11 + t_20, where tij = ∑kknak+ibn-k+j; hence cn+12-cncn+2=(t012-t00t02)+(t102-t00t20)+2(t01t10-t00t11). We will show that each of these parenthesized terms is nonnegative.

Let bj' = jnbj. Then the sequence 〈bj'〉 is log-concave; and ti0 is the (n + i)th term of the sequence ∑kakbn-k', which is log-concave by (f). Therefore t102 ≥t00t20. A similar argument shows that t012 ≥t00t02. Finally, Binet–Cauchy gives the identity
t01t10-t00t11=∑p<q(np)(nq)(ap+1aq-apaq+1)(bn-pbn-q+1-bn-p+1bn-q)

from the matrix product T = AXB, where Aij=ai+j,Xij=(jn)[i+j=n], Bij=bi+j. [D. W. Walkup, Journal of Applied Probability 13 (1976), 79–80.]

126. The stated probability is pm=(nm)mm(n-m)n-m/nn. We have pm/pm+1 = fm/fn−m−1, where fm = (m/(m + 1))m. Since f_0 ≥ f_1 > … , the minimum occurs when m = ⌊n/2⌋. And p⌊n/2⌋= (1 + O(1/n))/πn/2, by exercise 1.2.11.2–9.

127. (a) Random binary vectors have Pr(X_1 + … + Xn ≤ θn) ≤ x−θn ((1 + x)/2)n for 0 < x ≤ 1, by the tail inequality 1.2.10–(24). Set x = θ/(1 − θ) and multiply by 2n.

(b) We have lgθnn = Hθn-lg2πθ(1-θ)n + O(1/n) by 1.2.11.2–(18).

(c) Let pm′ m″ = Pr(x ⊕ X′ ⊕ X″ is sparse and νX′ = m′, νX′ = m′). We will prove that each pm′ m″ is exponentially small, using several instructive methods.

First, let ∈ = θ(1 − 2θ)/3. We can assume that (θ − ∈) n < m′, m″ ≤ θn, because PrvX' ≤ (θ-∈)n = O(n2Hθ-∈-Hθn) is exponentially small.

Second, let Y′ and Y″ be random binary vectors whose bits are independently 1 with probabilities m′/n and m″/n. Each bit of x ⊕ Y′ ⊕ Y″ is 1 with probability m′/n(1 − m″/n)+(1 − m′/n) m″/n ≥ 2(θ − )(1 − θ) ≥ θ + ∈ when x has a 0 bit, or (m′/n)(m″/n) + (1 − m″/n)(1 − m′/n) ≥ (θ − ∈ )2 + (1 − θ)2 ≥ θ + when x has a 1 bit. Therefore, by the tail inequality, we have Pr(x ⊕ Y′ ⊕ Y″ is sparse) ≤ αn, where α = (1+ /θ)θ(1 − /(1 − θ))1−θ. This is exponentially small, since α < 1.

Finally, let Z′ and Z″ be independent random bit vectors with νZ′ = m′ and νZ″ = m″. Then pm'm″ = mn,m″n/Sn,θ2Pm'm″, where Pm′m″ is the probability that x ⊕ Z′ ⊕ Z″ is sparse. Then Pr(x ⊕ Y′ ⊕ Y″ is sparse) ≥ Pr(x ⊕ Y′ ⊕ Y″ is sparse and νY″ = m′ and νY′ = m′) = Ω(Pm′m″/n) by exercise 126. (Study this!)

[V. Guruswami, J. Håstad, and S. Kopparty, IEEE Trans. IT-57 (2011), 718–725, used this result to prove the existence of efficient linear list-decodable codes.]

128. (a) Prk pings = kn1nk1-1nn-k is binomial, hence Pr1 ping = 1-1nn-1.

(b) It waits T rounds, where Pr(T = k) = (1 − p)k−1 p has the geometric distribution with p = 1n1-1nn-1. Hence, for example by exercise 3.4.1–17, we have ET = 1/p = nn/(n-1)n-1=(n-1)expn ln(1/(1-1/n)) - en-12e + O(1/n). (The standard deviation, en-12e-12 + O(1/n), is approximately the same as the mean.)

(c) The hint suggests that we study the “coupon collector’s distribution”: If each box of cereal randomly contains one of n different coupons, how many boxes must we buy before we’ve got every coupon? The generating function for this distribution is

Cz=nznn-1zn-z…zn-n-1z=nn/z-0n-1n/z-1…1n/z-n-1=nn/z-1, because the time to acquire the next coupon, after we’ve already got k of them, is a geometric distribution with generating function (n − k) z/(n − kz).

Let B be the number of boxes purchased. The upper tail inequality 1.2.10–(25) tells us that Pr(B ≥ (1 + ∈) n ln n) ≤ (n/(n − 1/2))−(1+∈)n ln n C(n/(n − 1/2)), which is
e1+∈n ln n ln1-1/2nnn-1/2=e-1+∈2 ln n+Olog nn4nn2n=πn-∈/21+Olog nn
by exercise 1.2.6–47. Thus B is a.s. less than (1 + ∈ ) n ln n.

Now let S be the number of successful accesses in r = ⌊(1 + ∈) en ln n⌋ rounds. Then S is equivalent to r tosses of a biased coin for which the probability of success is p=1-1nn-1 = 1/e + O 1/n, by (a). So S has the binomial distribution, and Pr(S ≤ (1 − ∈/2) rp) ≤ e−∈2 rp/8 by exercise 1.2.10–22(b). This argument proves that S is q.s. greater than (1 − ∈ /2) rp = (1 + ∈/2 − ∈ 2/2) n ln n + O(log n).

Consequently S attempts at coupon collecting will a.s. succeed.

(d) An argument similar to (c) applies, with ∈ ↦ − ∈ and n − 1/2 ↦ n + 1/2.

[This exercise is based on a protocol analyzed in Jon Kleinberg and Éva Tardos’s book Algorithm Design (Addison–Wesley, 2006), §13.1. See Uriel Feige and Jan Vondrák, Theory of Computing 6 (2010), 247–290, §3.1, for optimum contention resolution with a related (but different) model.]

129. The hint follows because | cot πz| ≤ (eπ +1)/(eπ −1) and | r(z)| = O(1/M^2)onthe path of integration. The function π cot πz has no finite singularities except for simple poles at k for all integers k. Furthermore its residue is 1 at each of its poles. Therefore ∑k=-∞∞ rk+∑j=1t (Residue of r(z) π cot πz at zj) = limM→∞ O(1/M) = 0.

Let the sums be S_1, S_2, S_3, S_4. We have S_1 = π^2/4, because the residue of (cot πz)/(2 z − 1)2 at 1/2 is − π/4. And S_2 = π coth π, because the residue of (cot πz)/(z^2 +1) at ± i is −(coth π)/2. Similarly, the residue of (cot πz)/(z^2 + z +1) at -1±i3/2 is − α, where α = tanh3π/2/3; hence S_3 = 2 πα. Finally, the residues of (cot πz)/((z^2 + z + 1)(2 z − 1)) at its poles are 27α1±i3/2 and 0; hence S4 = -27S3. (With hindsight, we can explain this “coincidence” by noting that 7/k2+k+12k-1 = 42k-1-2k+3k2+k+1 and that ∑k=-nn+1 12k-1=∑k=-nn-12k+1k2+k+1=0.)

130. (a) Clearly E X2 = 1π ∫-∞∞ t2 dt/1+t2 >2π∫1∞ dt, so E X^2 = ∞. But E X = 1π∫-∞∞ t dt/(1+t2)=12πln(1+∞2)-ln(1+(∞2))=∞-∞ is undefined. Thus X has no mean (although it does have the median value 0).

(b) 1/2, 2/3, and 5/6, because Pr|X|≤x=1π∫-xx dt1+t2=2π arctan x when x ≥ 0.

(c) This follows directly from the fact that Pr( X ≤ x) = (arctan x)/π +1/2.

(d) In step P4 of Algorithm 3.4.1P, V_1/V_2 is a random tangent, so it is a Cauchy deviate. Furthermore, by the theory underlying that algorithm, V_1/V_2 is the ratio of independent normal deviates; thus, Z ← X/Y is Cauchy whenever X and Y are independently normal. The Cauchy distribution is also Student’s t distribution with 1 degree of freedom; Section 3.4.1’s recipe for generating it is to compute Z ← X/| Y |.

(e) We have z ≤ Z ≤ z + dz ⇔ (z − qY)/p ≤ X ≤ (z + dz − qY)/p. Hence

Pr(z≤Z+dz⁢  and y ≤Y≤y+dy)=1πdzp1(1+(z-qy/p2))1πdy1+y2,
and we want to integrate this for −∞ <y < ∞. The integrand has poles at y = ± i and y = (z ± ip)/q, and it is O(1/M^4) when | y| = M. So we can integrate on a semicircular path, y = t for − M ≤ t ≤ M followed by y = Meit for 0 ≤ t ≤ π, obtaining

∫-∞∞dy(p2+(z-qy)2)(1+y2)=2πi((Residue at i)+(Residue at z+piq))=1p(1+z2)..

Thus Pr(z≤Z≤z+dz)=1πdz/(1+z2) as desired.

It follows by induction (see answer 42) that any convex combination of independent Cauchy deviates is a Cauchy deviate. In particular, the average of n independent Cauchy deviates is no more concentrated than a single deviate is; the “law of large numbers” doesn’t always hold. [S. D. Poisson proved this special case in Connaissance des Tems pour l’an 1827 (1824), 273–302. The distribution is named after A. L. Cauchy, not Poisson, because Cauchy clarified matters by publishing seven notes about it ― one note per week! ― in Comptes Rendus Acad. Sci. 37 (Paris, 1853), 64–68, ... , 381–385.]

(f) By (e), c · X is | c_1| + … + | cn| = ||c||1 times a Cauchy deviate. [This fact has important applications to dimension reduction and data streams; see P. Indyk, JACM 53 (2006), 307–323.]

(g) If t ≥ 0 we get e−t, using the residue of eitz/(1 + z^2) at z = i and the semicircular path of part (e), because the integrand is O(1/M^2) when | z| = M. If t ≤ 0 we can integrate in the opposite direction, getting e+t. Hence the answer is e−|t|.

131. (a) By exercise 129, c = 1/(π coth π). [Notice that coth π ≈ 1.0037 is nearly 1.]

(b) When n ≠ 0, the method of exercise 129 tells us, somewhat surprisingly, that ∑k=-∞∞ 1/1+k21+n-k2 = 2π coth π/n2 + 4. Thus Pr(X + Y = n) = 2 c/(n^2 + 4). When n is even, this is exactly 12 Pr2Z = n.

When n = 0, there’s a double pole and the calculations are trickier. We can more easily compute Pr(X+Y≠0)=∑n=1∞ 4cn2+4 = c(π coth 2π - 12)≈ .837717. Thus Pr(X + Y = 0) ≈ .162283.

132. (a) kKn-kN-K/nN. [Hence the probability generating function g(z)=Σk⁢  pk⁢  zk is a hypergeometric function, (nN-K)F(N-K−n+1-K,-N|z)/(nN); see Eq. 1.2.6–(39).]

(b) g′(1) = nK/N; {⌊((n + 1)(K +1) − 1)/(N +2)⌋, ⌊(n + 1)(K + 1))/(N +2)⌋}; n(N − n)(N − K)/(N^3 − N^2). (Note that g″(1) = n(n − 1) K(K − 1)/(N(N − 1)).)

(c) Let Q = X_1 + … + Xn and Zm = E(Q | X_1 ,...,Xm). Then we have Zm = (K − X_1 −‧‧‧− Xm)(n − m)/(N − m)+ X_1 + ‧‧‧ + Xm. The associated fair sequence is Ym = Zm − Zm−1 = Δm(X_1 + ‧‧‧ + Xm−1 − K) + cm Xm for 1 ≤ m ≤ n, where cm = (N − n)/(N − m) and Δm = cm − cm−1. Since Ym changes by at most cm when {X_1 ,..., Xm−1} are given and Xm varies, (37) tells us that Pr(Q ≥ nK/N + x) = Pr(Zn − Z_0 ≥ x) = Pr(Y_1 + … + Yn ≥ x) ≤ e−2x2/(c12+⋅⋅⋅+cn2)≤e−2x2/n.

133. (a) By induction on m: Suppose m > 1 and no t rows are shattered. Discard duplicate columns, and let 2b of the remaining ones have a “mate” whose bit in the bottom row is complemented. Let a of them have no mate. Then the first m − 1 rows contain a + b ≤ f(m − 1, t) distinct columns, by induction; and b ≤ f(m − 1, t − 1). Hence there are a + 2b ≤ f(m − 1, t)+ f(m − 1, t − 1) = f(m, t) distinct columns.

(b) For example, let the columns be all length- m vectors that have at most t−1 1s.

[N. Sauer, Journal of Combinatorial Theory A13 (1972), 143–145.]

134. (a) Use Chebyshev’s inequality (18), because the variance is pj(1 − pj) ≤ 1/4.

(b) Consider the m2m equally likely ways we could have gotten two samples (χ, χ′) from the same 2m atomic events. If Aj occurs K = Mj(χ)+ Mj(χ′) times,

Pr (Ej^(χ,χ')>∈) = Pr(∑{(kK)(m-k2m-k)/(m2m)| | k-K/2| >∈m})≤2e-2(∈m)2/m.

(c) Δ2m (A) PrE^jχ,χ'>∈/2≥Prmaxj E^jχ,χ'≥∈/2 and E(χ) > ∊) ≥ Pr (Ej(χ′) ≤ ∊/2 and Ej(χ) > ∊ and E(χ)>∈)≥12Pr(Ej(χ)>∈ and E(χ) > ∊). [Teoriya Veroyatnosteĭ i ee Primeneniîa 16 (1971), 264–279.]

135. (Notice that the smallest non-Baxter permutations are 3142 and its inverse, 2413.)

If P is a Baxter permutation, so are PR = pn ... p_1 and PC=p¯1...p¯n, where x¯=n+1−x. So is the permutation P\n obtained by deleting n; and so are the permutations P\x obtained by deleting x and subtracting 1 from each element that exceeds x, if x = pn or x = 1 or x = p_1. (Consider, for example, deleting n from P−.)

Let’s look at the n+1 permutations obtained by inserting n+1 into an n-element Baxter permutation. For example, when n = 8 and P = 21836745 the nine extensions are 921836745, 291836745, 219836745, 218936745, 218396745, 218369745, 218367945, 218367495, 218367459. Only four of these fail Baxter’s property, namely 291836745, 218396745, 218369745, and 218367495; and we soon discover the general rule: n + 1 can be Baxterly inserted if and only if it’s placed just before a left-to-right maximum, or just after a right-to-left maximum. (In our example, the left-to-right maxima are 2 and 8; the right-to-left maxima are 5, 7, and 8.)

Let Bn(i, j, k) be the number of (n+1)-element Baxter permutations with exactly i + 1 left-to-right maxima, j + 1 left-to-right minima, k ascents, and n − k descents. Such permutations correspond to floorplans with n + 1 rooms, i + 1 rooms touching the bottom of the frame, j + 1 rooms touching the left of the frame, k + 2 vertical bounds, and n − k + 2 horizontal bounds (see exercise 7.2.2.1–372). The reasoning above yields the interesting recurrence

B1(i,j,k) = [i = j = k= 0], Bn+1(i+1, j+1, k)= Σi′>i Bn(i', j, k)+Σj′>j Bn(i,j', k-1);
and the solution can be expressed as a determinant of binomial coefficients:

Bn(i,j,k)=det ((n−j−1k−1)(nk−1)(n-i-1n-k+1)(n−j−1k)(nk)(n−i−n−k)(n−j−1k+1)(nk+1)(n−i−1n−k−1)), unless {i = 0 and j=nori = n and j=0.

Summing on i and j now gives the simpler formula
bn(k)=tn+1(k+1)/tn+1(1),where⁢  tn(k) = (nk−1)(nk)(nk+1),,
for the number of n-element Baxter permutations with exactly k ascents.

Since the terms with k ≈ n/2 dominate the sum bn = Σk bn(k), we obtain the asymptotic value
bn = 8n+212πn41-223n+On-2,
due to A. M. Odlyzko. [See G. Baxter, Proc. American Math. Soc. 15 (1964), 851– 855; F. R. K. Chung, R. L. Graham, V. E. Hoggatt, Jr., and M. Kleiman, Journal of Combinatorial Theory A24 (1978), 382–394; W. M. Boyce, Houston J. Math. 7 (1981), 175–189; S. Dulucq and O. Guibert, Discrete Math. 180 (1998), 143–156.] R. L. Ollerton has found the recurrence (n + 2)(n + 3) bn = (7 n^2 + 7 n − 2) bn−1 + 8(n − 1)(n − 2) bn−2, with b_1 = 1, as well as the closed form bn = F2,31-n,-n,-1-n|-1.

The initial terms are (b_0, b_1, ...) = (1, 1, 2, 6, 22, 92, 422, 2074, 10754, 58202, ...).

136. It’s true if y≤x+12, because f(x + t) − f(x) increases from f(t) to − f(1 − t) as x increases from 0 to 1 − t. But it fails when x<12 and y = 1.

137. (a) The sets U = x | Pr X≤x≥12 and L = x | Pr X<x≤12 are intervals. Let m = inf U and m = sup L; then U ∩ L = m..m. Since the distribution function Pr(X ≤ x) is right-continuous, m−∈U; similarly, m−∈L, because Pr(X<x) is left-continuous. Also m ≤ m; for if  m < x < m then Pr(X≤x)<12<Pr(X<x).

(b) If m<m then PrX ≤ m≤ PrX <m = 1 -PrX ≥ m≤12≤PrX ≤ m.

(c) PrX ≤ y≥12 implies y ≥m; Pr(X < x) ≤ 12 implies x ≤ m; so it’s true if m = m. But we might have x > m or y < m.

(d) Suppose m ∊ med X and c < m. (A similar argument applies when c > m.) Let Δ x = | x − c|−| x − m|. If x ≥ m we have Δ x = m − c. If x < m we have ∆x = c-m+2x-˙c; hence E(Δ X | X < m) ≥ c− m. Therefore E(Δ X) ≥ (c− m) Pr(X<m)+ (m − c)Pr(X ≥ m) = (m − c) (2 Pr(X ≥ m) − 1) ≥ 0. Equality holds if and only if PrX ≥ m = 12 and Pr(c<X <m) = 0; the latter is the same as Pr(X ≤ c) = Pr(X<m). [See M. Mitzenmacher and E. Upfal, Probability and Computing (2017), Theorem 3.9.]

(e) True by Cantelli’s inequality, answer 48: If m ≥ μ then 12≤Pr(X≥m)≤σ2/(σ2+(m−μ)2). If 12≤Pr(−X≥−m)≤σ2/(σ2+(μ−m)2).

(f) Call f a “C-function” if It = { x | f(x) ≤ t} is connected and closed for all t. Every convex f is a C-function; for if a ∊ It and b ∊ It, we have pa +(1 − p) b ∊ It for 0 ≤ p ≤ 1; also It is closed because f is continuous. (There also are rather wild C-functions, such as fx = x < 0? 3: x <1? 2-x: x≤2? x-1: x≤3? x: x..

Given a C-function f and a random variable X, let med X = m..m and med fX = M..M. If M≤ M ≤M, then IM is a closed interval and Pr(X ∊ IM) = Pr(f(X)≤ M) ≥ 12. Thus by (c), either fm≤M or fm≤M. (For example, if f(x) = − x we have m = -M and m = -M.) [See M. Merkle, Statistics & Probability Letters 71 (2005), 277–281.]

138. Working in the slices of probability space where Y is constant, we have (by definition) var(X | Y) = E(X^2 | Y) − (E(X | Y))2 and var(E(X | Y)) = E(E(X | Y))2 − (E(E(X | Y)))2. Hence E(var(X | Y)) = E(E(X^2 | Y)) − E(E(X | Y))2. The complicated term E(E(X | Y))2 fortuitously cancels out, giving var(E(X | Y)) + E(var(X | Y)) = E(E(X^2 | Y)) − (E(E(X | Y)))2 = E X^2 − (E X)2. [See CMath, pages 423–425.]

139. Let x(z) = Σk Pr(Xn= k) zk, gn(w, z) = Σj,k Pr(Rn = j, Sn= k) wj zk, hn(w, z) = Σj,k PrSn+ = j, Sn = kwjzk; these generating functions involve negative values of k, so we treat them as “formal series.” We shall prove that g = h, where
g = ∑n=0∞ gn w, ztnandh = exp ∑n=1∞hnw, ztnn;
that will suffice because gn(w, z) = rn(wz, z−1) and hn w,z=sn+wz+sn-z-1-1.

Let X be the operation that multiplies a formal series by x(z), and let P be the operation that replaces wj zk by wmax(j, k) zk. Notice that hn(w, z) = P (x(z)n); furthermore we have g_0(w, z) = 1, gn(w, z) = PXgn−1 for n > 0. It follows that g is the unique formal series that satisfies g = 1 + tP Xg. To finish the proof, we have 1-tXh = exp 1-P ln 1-txz=1+∑n=1∞ 1-Pln1-txzn/n!; hence h − tP Xh = P ((1 − tX) h) = 1. [J. G. Wendel, Proc. Amer. Math. Soc. 9 (1958), 905–908.]

140. (a) Let q = 1 − p. The expected number of marked elements that remain a max is ∑kknpkqn-k Hk = Hn + ln p+Oqn/n, by Theorem 1.2.7A. To this we add ∑m=1n tm, where tm = Pr(xm unmarked and still a max) = ∑j,kjm-1kn-mpj+kqn-j-k/jj+k. (For example, t_1 = q; t_2 = q^2 + (q − qn)/(n − 1); tm = tn + 1−m.) The identity ∑k knpkqn-k/jk+j = 1-∑k=0j=1 kn+jpkqn+j-k/pjjn+j shows that, for fixed m, tm = ∑j=0m-1 qm-jm-1j / n-m+jj + O qn/n. Summing on m, and trading tails, yields t_1 + … + tn = 2 q/p + O(qn/n). [For this result, as well as those of parts (b) and (c), see C. Banderier, R. Beier, and K. Mehlhorn, LNCS 2747 (2003), 198–207.]

(b) Say m = n. If a of the first m elements are marked, and b of the last n− m, the probability that all   a leave the first m positions is q=ba/a+ba > b-a/ba; and in this case λ(X) ≥ m − a. We q.s. have a ≤ 34m and b ≥14n + m; consequently q ≥ exp(a ln(1 − a/b)) ≥ exp(− a^2/(b − a)) ≥ exp(−9/4) and λ(X)=Ω(n).

(c) Let m = 8n/pln n and ignore all xk with k ≤ m or xk ≥ n − m; at most 2 m maxs are ignored. At most about ln pn of the marked elements are maxs. And if xk is neither ignored nor marked, it’s a max with probability O(1/n); the reason is that q.s. at most 2 pn are marked, of which ≥ pm/2 precede xk and ≥ pm/2 exceed xk.

(d) If x¯k>x¯k+1, swapping x¯k↔x¯k+1 and δk ↔ δk+1 doesn’t decrease E λ(X).

(e) Let m=∈n and ∆k=x¯k-x¯k-m, where x¯k=0 for k < 0. If xk is a max then either (*) ∈ < Δk + δk or (**)(∈ ≥ Δk + δk > max{ δk−1 ,..., δk−m}. One can show that Pr(*) ≤ Pr(**); hence Pr(xk is a max) ≤ Δk/(2 ∈)+1/(m +1)+1/k. Sum on k.

See V. Damerow, B. Manthey, F. Meyer auf der Heide, H. Räcke, C. Scheideler, C. Sohler, and T. Tantau, ACM Transactions on Algorithms 8 (2012), 30:1–30:28, where a matching lower bound is also proved. Similarly, if each δk is a normal deviate with standard deviation σ, they showed that EλX = O log n 1+σ-1log n.

141. We can assume that p_1 + … + pn = 1. Then eln(E X) ≥ eE ln X (ln is concave).

142. (a) Let pj = Pr(| X|= xj). Since every term in the difference MqMt - MrMs = ∑j<kpjpkxjqxkqxks-q-xjs-qxkr-q-xjr-q is nonnegative, we have Mq Mt ≥ Mr Ms.

(b) The hint gives Ms/MrMr/s-r≥x1p1x1r⋯xnpnxnr. Similarly, but reversing the inequality because q<r, Mq/MrMr/q-r≤x1p1x1r⋯xnpnxnr. Take the Mrth root.

(c) The “fact” follows when (q, r, s) = (0 , 1/p, 1) in (b). Let c = 1/∑bkq, and set pk = cbkq, xk=akq/bkq; then M1/p = c Σ ak bk, M1 = c∑ akp. (When 0 < p < 1 and q < 0, the same relation holds but with ≤ changed to ≥ and bk = 0 forbidden.)

(d) |E X Y | ≤ E (|X||Y|) = ∑i,j pij1/p+1/qxiyj ≤(E|X|p)1/p(E|Y|q)1/q, where pij = Pr(| X|= xi and | Y |= yj) is the joint distribution of | X| and | Y |.

Historical notes: This inequality and Jensen’s inequality evolved in concert. The fact that E | X|r ≤ (E | X|)r for 0 < r < 1, while E | X|r ≥ (E | X|)r for other values of r, was already implicit on page 155 of Reynaud and Duhamel’s Problèmes et développemens (Paris: 1823). Rogers published his contributions in Messenger of Math. 17 (1887), 145–150 (with a few typographic errors). That inspired O. Höder [Göttinger Nachrichten (1889), 38–47] to prove (20) for all f with f″(x) ≥ 0, obtaining Rogers’s identities as corollaries. Many related results are detailed in Hardy, Littlewood, and Pólya’s book Inequalities (1934), Chapter 2. For example, if pj ,aij ≥ 0 for 1 ≤ i ≤ m and 1 ≤ j ≤ n, with Σpj = 1, their Theorem 10 states that

∑i=1m(Πj=1naijpj)≤Πj=1n(∑i=1maij)pj.[The⁢ case⁢  n=2, p1=1p,p2=1q,is(c).]

143. Let M = (E(| X| + | Y|)p)1/p = (Σi,j (pij(xi + yj)p))1/p, with the pij of answer 142(d). Then we have M = Σ(x)+Σ(y), where Σ(x) = Σi,j pij xi(xi + yj)p−1/Mp−1 = ∑i,jpij1/pxipij1/pxi+yip-1/Mp-1≤∑i,jpij xip1/p∑ijpijxi+yjp1/q/Mp-1 = E|X|p1/p. Add Σ(y). [H. Minkowski, Geometrie der Zahlen (Leipzig, 1896), §40(I).]

144. (a) By convexity, | x|p = | E(x + Y)|p ≤ E | x + Y |p for any x. Take E of both sides.

(b) By (a), E | X|p = E | X+|p ≤ E | X+ − X−|p.

(c) The hint follows because (1 + x)p + (1 − x)p − 2 xp ≥ 2 for 0 ≤ x ≤ 1. Consequently E | X|p + E | Y |p ≤ E | X + Y |p when E | X + Y |p = E | X − Y |p. Now use induction on n. [See J. A. Clarkson, Trans. Amer. Math. Soc. 40 (1936), 396–414.]

(d) E|X1|p+…+E|Xn|p ≤ E| X1sym|p+…+E| X1sym+…+Xnsym|p=E|X1+…+Xn+-X1-+…+Xn-|p ≤ E2p-1| X1++…+Xn+|p+E2p-1|-(X1-+…+Xn-)|p=2p E|X1+…+Xn|p. [See A. Gut, Probability: A Graduate Course (Springer, 2013), Theorem 3.6.1. We’ve used the fact that | x + y|p ≤ 2p−1(| x|p + | y|p), which actually holds for p ≥ 1 because the mapping x ↦ | x|p is convex.]

145. We have a12+⋯+an2m = ∑k1,…,kn ck1,…kn a12k1 …an2kn by the multinomial theorem, Eq. 1.2.6−(42), where; ck1,…kn = k1,…,knm; E((a1X1+⋅⋅⋅+anXn)2m)=∑k1,…,kn c'(k1,…kn) a1k1…ankn, where c'k1,…,kn = k1,…,kn2m when each kj is even, otherwise c′(k_1,...,kn) = 0. And c′(2k_1,...,2kn)/c(k_1,...,kn) = (2 m − 1)!!/ ∏j=1m2kj-1!!. [A. Khintchine, Math. Zeitschrift 18 (1923), 109–116. More generally, (a12+…+an2)p/2 ≤ E|a1X1+…+anXn|p≤2p/2π-1/2Γ(p+12)(a12+…+an2)p/2, for all p ≥ 2; see U. Haagerup, Studia Mathematica 70 (1981–1982), 231–283.]

146. For every binary vector t = t_1 ... tn let Tnt = ∑k=1n-1tk Xk. Also let ∑k=1n Xk, Snsym = ∑k=1n Xksym, Tnsym t = ∑k=1n-1tk Xksym. By exercise 144 we have

2−2mE Tn(t)2m≤2−2mE Tnsym(t)2m=2−2m⁢E (Snsym)2m≤E Sn2m                                                                                  ≤E (Tnsym)2m=E Tnsym(t)2m≤22m⁢ E Tn(t)2m
for all t, because Snsym and Tnsym t have the same distribution. Exercise 145 tells us that

∑k=1n xk2m ≤ 12n∑t∑k=1n-1tkxk2m ≤2m - 1!!∑k=1nxk2m
for all sequences x_1 ...xn of atomic values. So the result follows by applying E. [Fundamenta Mathematicæ 29 (1937), 60–90; Studia Mathematica 7 (1938), 104–120.]

147. This is an application of the previous several exercises; see A. Gut, Probability (2013), Theorem 3.9.1. [H. P. Rosenthal, Israel J. Mathematics 8 (1970), 273–303.]

As to volume 4, well, I’m making progress but it is the toughest of the lot.

— DONALD E. KNUTH, letter to Michael F. Yoder (19 November 1973)

Section 7.2.2

1. Although many formulations are possible, the following may be the nicest: (i) Dk is arbitrary (but hopefully finite), and Pl is always true. (ii) Dk = {1, 2,...,n} and Pl = ‘xj ≠ xk for 1 ≤ j < k ≤ l’. (iii) For combinations of n things from N, Dk = {1,...,N + 1 − k} and Pl = ‘x_1 > ··· > xl’. (iv) Dk = {0, 1,..., ⌊n/k⌋}; Pl = ‘x_1 ≥ ··· ≥ xl and n − (n − l)xl ≤ x_1 + ··· + xl ≤ n’. (v) For restricted growth strings, Dk = {0,...,k−1} and Pl = ‘xj+1 ≤ 1+max(x_1,...,xj) for 1 ≤ j < l’. (vi) For indices of left parentheses (see 7.2.1.6 (8)), Dk = {1,...,2k − 1} and Pl = ‘x_1 < ··· < xl’.

2. True. (If not, set D_1 ← D_1 ∩ {x | P_1(x)}.)

3. Let Dk = {1,...,max degree on level k − 1}, and let Pl(x_1,...,xl) = ‘x_1. ··· .xl is a label in T’s Dewey decimal notation’ (see Section 2.3).

4. We can restrict D_1 to {1, 2, 3, 4}, because the reflection (9−x_1) ... (9−x_8) of every solution x_1 ... x_8 is also a solution. (H. C. Schumacher made this observation in a letter to C. F. Gauss, 24 September 1850.) Notice that Fig. 68 is left-right symmetric.

5. try (l) = “If l > n, visit x_1 ... xn. Otherwise, for xl ← min Dl, min Dl + 1, ... , max Dl, if Pl(x_1,...,xl) call try (l + 1).”

This formulation is elegant, and fine for simple problems. But it doesn’t give any clue about why the method is called “backtrack”! Nor does it yield efficient code for important problems whose inner loop is performed billions of times. We will see that the key to efficient backtracking is to provide good ways to update and downdate the data structures that speed up the testing of property Pl. The overhead of recursion can get in the way, and the actual iterative structure of Algorithm B isn’t difficult to grasp.

6. Excluding cases with j = r or k = r from (3) yields respectively (312, 396, 430, 458, 458, 430, 396, 312) solutions. (With column r also omitted there are just (40, 46, 42, 80, 80, 42, 46, 40).)

7. Yes, almost surely for all n > 16. One such is x_1x_2 ... x_17 = 2 17 12 10 7 14 3 5 9 13 15 4 11 8 6 1 16. [See Proc. Edinburgh Math. Soc. 8 (1890), 43 and Fig. 52.] Preuβer and Engelhardt found 34,651,355,392 solutions when n = 27.

8. Yes: (42736815, 42736851); also therefore (57263148, 57263184).

9. Yes, at least when m = 4; e.g., x_1 ... x_16 = 5 8 13 16 3 7 15 11 6 2 10 14 1 4 9 12. There are no solutions when m = 5, but 7 10 13 20 17 24 3 6 23 11 16 21 4 9 14 2 19 22 1 8 5 12 15 18 works for m = 6. (Are there solutions for all even m ≥ 4? C. F. de Jaenisch, Traité des applications de l’analyse mathématique au jeu des échecs 2 (1862), 132 133, noted that all 8-queen solutions have four of each color. He proved that the number of white queens must be even, because   Σk=14m(xk+k) is even.)

10. Let bit vectors al, bl, cl represent the “useful” elements of the sets in (6), with al = ∑{2x−1 | x ∈ Al}, bl = ∑{2x−1 | x ∈ Bl ∩ [1 .. n]}, cl = ∑{2x−1 | x ∈ Cl ∩ [1 .. n]}. Then step W2 sets bit vector sl←μ&a¯l&b¯l&c¯l, where μ is the mask 2n − 1.

In step W3 we can set t ← sl & (−sl), al+1 ← al + t, bl+1 ← (bl + t) ≫ 1, cl+1 ← ((cl + t) ≪ 1) & μ; and it’s also convenient to set sl ← sl − t at this time, instead of deferring this change to step W4.

(There’s no need to store xl in memory, or even to compute xl in step W3 as an integer in [1 .. n], because xl can be deduced from al − al−1 when a solution is found.)

11. (a) Only when n = 1, because reflected queens can capture each other.

(b) Queens not in the center must appear in groups of four.

(c) The four queens occupy the same rows, columns, and diagonals in both cases.

(d) In each solution counted by cn we can independently tilt (or not) each of the ⌊n/4⌋ groups of four. [Mathematische Unterhaltungen und Spiele 1, second edition (Leipzig: Teubner, 1910), 249–258.]

12. With distinct xk, Σk=1n(xk+k)=2(n+12)≡0 (modulo n). If the (xk + k) mod n are also distinct, the same sum is also ≡(n+12). But that’s impossible when n is even.

Now suppose further that the numbers (xk − k) mod n are distinct. Then we have Σk=1n(xk+k)2≡Σk=1n(xk−k)2≡Σk=1nk2=n(n+1)(2n+1)/6. And we also have Σk=1n(xk+k)2+Σk=1n(xk−k)2=4n(n+1)(2n+1)/6≡2n/3, which is impossible when n is a multiple of 3. [See W. Ahrens, Mathematische Unterhaltungen und Spiele 2, second edition (1918), 364–366, where G. Pólya cites a more general result of A. Hurwitz that applies to wraparound diagonals of other slopes.]

Conversely, if n isn’t divisible by 2 or 3, we can let xn = n and xk = (2k) mod n for 1 ≤ k ≤ n. [The rule xk = (3k) mod n also works. See Édouard Lucas, Récréations Mathématiques 1 (1882), 84–86.]

13. The (n + 1) queens problem clearly has a solution with a queen in a corner if and only if the n queens problem has a solution with a queen-free main diagonal. Hence by the previous answer there’s always a solution when n mod 6 ∈ {0, 1, 4, 5}.

Another nice solution was found by J. Franel [L’Intermédiaire des Mathématiciens 1 (1894), 140–141] when n mod 6 ∈ {2, 4}: Let xk = (n/2 + 2k − 3[2k ≤ n]) mod n + 1, for 1 ≤ k ≤ n. With this setup we find that xk − xj = ±(k − j) and 1 ≤ j < k ≤ n implies (1 or 3)(k − j) + (0 or 3) ≡ 0 (modulo n); hence k − j = n − (1 or 3). But the values of x_1, x_2, x_3, xn − 2, xn − 1, xn give no attacking queens except when n = 2.

Franel’s solution has empty diagonals, so it provides solutions also for n mod 6 ∈ {3, 5}. We conclude that only n = 2 and n = 3 are impossible.

[A more complicated construction for all n > 3 had been given earlier by E. Pauls, in Deutsche Schachzeitung 29 (1874), 129–134, 257–267. Pauls also explained how to find all solutions, in principle, by building the tree level by level (not backtracking).]

14. For 1 ≤ j ≤ n, let x1(j)…xm(j) be a solution for m queens, and let y_1 . . . yn be a solution for n toroidal queens. Then X(i−1)n+j=(xi(j)−1)n+yj (for 1 ≤ i ≤ m and 1 ≤ j ≤ n) is a solution for mn queens. [I. Rivin, I. Vardi, and P. Zimmermann, AMM 101 (1994), 629–639, Theorem 2.]

15. More precisely, there’s a constant σ = e1−α such that, for any fixed ϵ with 0 < ϵ < σ, Q(n)/n! is q.s. between ((1 − ϵ)σ)n and ((1 + ϵ)σ)n. In fact, a subtle analysis [arXiv:2107.13460 [math.CO] (2021), 51 pages] shows that the average of all solutions approaches a fascinating probability distribution. P. Nobel, A. Agrawal, and S. Boyd have computed α accurately [arXiv:2112.03336 [math.CO] (2021), 14 pages].

16. Let the queen in row k be in cell k. Then we have a “relaxation” of the n queens problem, with |xk − xj| becoming just xk − xj in (3); so we can ignore the b vector in Algorithm B* or in exercise 10. We get
n=01234567891011121314H(n)=11137238340521131265782297596483469865540071743367854835

[N. J. Cavenagh and I. M. Wanless, Discr. Appl. Math. 158 (2010), 136.146, Table 2.]

17. It fails spectacularly in step L5. The minus signs, which mark decisions that were previously forced, are crucial tags for backtracking.

18. x4…x8=2¯1¯04¯0, p_0 … p_4 = 33300, and y_1y_2y_3 = 130. (If xi ≤ 0 the algorithm will never look at yi; hence the current state of y_4 … y_8 is irrelevant. But y_4y_5 happens to be 20, because of past history; y_6, y_7, and y_8 haven’t yet been touched.)

19. We could say Dl is {−n,...,−2, −1, 1, 2,...,n}, or {k | k ≠ 0 and 2 − l ≤ k ≤ 2n − l − 1}, or anything in between. (But this observation isn’t very useful.)

20. First we add a Boolean array a_1 ... an, where ak means “k has appeared,” as in Algorithm B*. It’s 0 ... 0 in step L1; we set ak ← 1 in step L3, ak ← 0 in step L5.

The loop in step L2 becomes “while xl < 0, go to L5 if l ≥ n − 1 and a2n−l−1 = 0, otherwise set l ← l + 1.” After finding l + k + 1 ≤ 2n in L3, and before testing xl+k+1 for 0, insert this: “If l ≥ n − 1 and a2n−l−1 = 0, while l + k + 1 ≠ 2n set j ← k, k ← pk.”

21. (a) In any solution xk=n⇔xk+n+1=−n⇔xn−kD=n.

(b) xk = n − 1 for some k ≤ n/2 if and only if xkD=n−1 for some k > n/2.

(c) Let n′ = n − [n is even]. Change ‘l ≥ n − 1 and a2n−l−1 = 0’ in the modified step L2 to ‘(l = ⌊n/2⌋ and an′ = 0) or (l ≥ n − 1 and a2n−l−1 = 0)’. Insert the following before the other insertion into step L3: “If l = ⌊n/2⌋ and an′ = 0, while k ≠ n′ set j ← k, k ← pk.” And in step L5 — this subtle detail is needed when n is even — go to L5 instead of L4 if l = ⌊n/2⌋ and k = n′.

22. The solutions 11¯ and 211¯2¯ for n = 1 and n = 2 are self-dual; the solutions for n = 4 and n = 5 are 4311¯23¯4¯2¯,2452¯311¯4¯3¯5¯,4511¯234¯2¯5¯3¯, and their duals. The total number of solutions for n = 1, 2, ... is 1, 1, 0, 2, 4, 20, 0, 156, 516, 2008, 0, 52536, 297800, 1767792, 0, 75678864, ... ; there are none when n mod 4 = 3, by a parity argument.

Algorithm L needs only obvious changes. To compute solutions by a streamlined method like exercise 21, use n′ = n − (0, 1, 2, 0) and substitute ‘l = ⌊n/4⌋ + (0, 1, 2, 1)’ for ‘l = ⌊n/2⌋’, when n mod 4 = (0, 1, 2, 3); also replace ‘l ≥ n − 1 and a2n−l−1 = 0’ by ‘l ≥ ⌈n/2⌉ and a⌊(4n+2−2l)/3⌋ = 0’. The case n = 15 is proved impossible with 397 million nodes and 9.93 gigamems.

23. slums → sluff, slump, slurs, slurp, or sluts; (slums, total) → (slams, tonal).

24. Build the list of 5-letter words and the trie of 6-letter words in step B1; also set a_01a_02a_03a_04a_05 ← 00000. Use min Dl = 1 in step B2 and max Dl = 5757 in step B4. To test Pl in step B3, if word xl is c_1c_2c_3c_4c_5, form al1 ... al5, where alk = trie [a(l−1)k, ck] for 1 ≤ k ≤ 5; but jump to B4 if any alk is zero.

25. There are 5 × 26 singly linked lists, accessed from pointers hkc, all initially zero. The xth word cx1cx2cx3cx4cx5, for 1 ≤ x ≤ 5757, belongs to 5 lists and has five pointers lx1lx2lx3lx4lx5. To insert it, set lxk ← hkcxk, hkcxk ← x, and skcxk ← skcxk + 1, for 1 ≤ k ≤ 5. (Thus skc will be the length of the list accessed from hkc.)

We can store a “signature” Σc=126 2c−1[trie[a,c]≠0] with each node a of the trie. For example, the signature for node 260 is 2^0 + 2^4 + 2^8 + 2^14 + 2^17 + 2^20 + 2^24 = #1124111, according to (11); here A ↔ 1, ... , Z ↔ 26.

The process of running through all x that match a given signature y with respect to position z, as needed in steps B2 and B4, now takes the following form: (i) Set i ← 0. (ii) While 2i & y = 0, set i ← i + 1. (iii) Set x ← hz(i+1); go to (vi) if x = 0. (iv) Visit x. (v) Set x ← lxz; go to (iv) if x ≠ 0. (vi) Set i ← i + 1; go to (ii) if 2i ≤ y.

Let trie [a, 0] be the signature of node a. We choose z and y = trie [a(l−1)z, 0] in step B2 so that the number of nodes to visit, Σc=126 szc[2c−1&y≠0], is minimum for 1 ≤ z ≤ 5. For example, when l = 3, x_1 = 1446, and x_2 = 185 as in (10), that sum for z = 1 is s_11 + s_15 + s_19 + s1(15) + s1(18) + s1(21) + s1(25) = 296 + 129 + 74 + 108 + 268 + 75 + 47 = 997; and the sums for z = 2, 3, 4, 5 are 4722, 1370, 5057, and 1646. Hence we choose z = 1 and y = #1124111; only 997 words, not 5757, need be tested for x_3.

The values yl and zl are maintained for use in backtracking. (In practice we keep x, y, and z in registers during most of the computation. Then we set xl ← x, yl ← y, zl ← z before increasing l ← l + 1 in step B3; and we set x ← xl, y ← yl, z ← zl in step B5. We also keep i in a register, while traversing the sublists as above; this value is restored in step B5 by setting it to the zth letter of word x, decreased by ’A’.)

26. Here are the author’s favorite 5 × 7 and 5 × 8, and the only 5 × 9’s:

S M A S H E S

P A R T I A L

I M M E N S E

E M E R G E D

S A D N E S S

G R A N D E S T

R E N O U N C E

E P I S O D E S

B A S E M E N T

E Y E S O R E S

P A S T E L I S T

A C C I D E N C E

M O R T G A G O R

P R O R E F O R M

A N D E S Y T E S

V A R I S T O R S

A G E N T I V A L

C O E L O M A T E

U N D E L E T E D

O Y S T E R E R S

No 5 × 10 word rectangles exist, according to our ground rules.

27. (1, 15727, 8072679, 630967290, 90962081, 625415) and (15727.0, 4321.6, 1749.7, 450.4, 286.0). Total time ≈ 18.3 teramems.

28. Build a separate trie for the m-letter words; but instead of having trie nodes of size 26 as in (11), it’s better to convert this trie into a compressed representation that omits the zeros. For example, the compressed representation of the node for prefix ‘CORNE’ in (12) consists of five consecutively stored pairs of entries (‘T’, 5013), (‘R’, 171), (‘L’, 9602), (‘D’, 3878), (‘A’, 3879), followed by (0, 0). Similarly, each shorter prefix with c descendants is represented by c consecutive pairs (character, link), followed by (0, 0) to mark the end of the node. Steps B3 and B4 are now very convenient.

Level l corresponds to row il = 1 + (l − 1) mod m and column jl = 1 + ⌊(l − 1)/m⌋. For backtracking we store the n-trie pointer ail,jl as before, together with an index xl into the compressed m-trie.

This method was used by M. D. McIlroy in 1975 (see answer 32). It finds all 5 × 6 word rectangles in just 400 gigamems; and its running time for “transposed” 6 × 5 rectangles turns out to be slightly less (380 gigamems). Notice that only one mem is needed to access each (character, link) pair in the compressed trie.

29. Yes, exactly 1618 of the 625415 solutions have repeated words. For example:

A C C E S S

M O O L A H

I M M U N E

N E E D E D

O T T E R S

A S S E R T

J A I L E R

U G L I F Y

G E O D E S

A S S E R T

B E G G E D

R E A L E R

A R T E R Y

W I E N I E

L E S S E R

M A G M A S

O N L I N E

D I O X I N

A S S E S S

L E S S E E

T R A D E S

R E V I S E

O T I O S E

T R A D E S

H O N E S T

30. The use of a single compressed trie both horizontally and vertically leads to a very pretty algorithm, which needs only 120 Mμ to find all 541,968 solutions. De Morgan’s example isn’t among them, because the proper name ‘ELLEN’ doesn’t qualify as a word by our conventions. But some of the squares might be “meaningful,” at least poetically:

B L A S T

L U N C H

A N G E R

S C E N E

T H R E E

W E E K S

E V E N T

E E R I E

K N I F E

S T E E L

T R A D E

R U L E D

A L O N G

D E N S E

E D G E S

S A F E R

A G I L E

F I X E S

E L E C T

R E S T S

A D M I T

D R O N E

M O V E S

I N E P T

T E S T S

Y A R D S

A P A R T

R A D I I

D R I L L

S T I L L

Just six of the solutions belong to the restricted vocabulary WORDS(500); three of them actually belong to WORDS(372), namely **ASS|*IGHT|AGREE|SHEEP|STEPS, where *** is either CLL or GLL or GRR. (And *** = GRL gives an unsymmetric 5 × 5 in WORDS(372). There are (1787056 − 541968)/2 = 622544 unsymmetric squares in WORDS(5757).)

31. Yes, 27 of them. The search is greatly facilitated by noting that the NE-to-SW diagonal word must be one of the 18 palindromes in WORDS(5757). ‘SCABS|CANAL|ANGLE|BALED|SLEDS’, which belongs to WORDS(3025), has the most common words. [See the end of Chapter 18 in Babbage’s Passages from The Life of a Philosopher (London: 1864).]

32. There are (717, 120386, 2784632, 6571160, 1117161, 13077, 6) of sizes 2 × 2, ... , 8 × 8, and none larger than this. Each of these runs needed fewer than 6 gigamems of computation. Example solutions with words as common as possible are

with the following numeric ranks of “minimax rarity” within their lists: TO = 2, SEE = 25, AREA = 86, ERROR = 438, ESTEEM = 1607, TREBLES = 5696, ETERNISE = 23623.

[Word squares go back thousands of years; ‘SATOR|AREPO|TENET|OPERA|ROTAS’, a famous 5 × 5 example that is found in many places including the ruins of Pompeii, actually has fourfold symmetry. But 6 × 6 squares appear to have been unknown until William Winthrop, the U.S. consul in Malta(!), published ‘CIRCLE|ICARUS| RAREST|CREATE|LUSTRE|ESTEEM’ in Notes & Queries (2) 8 (2 July 1859), page 8, claiming to have thereby “squared the circle.” (If he had been told not to use a proper name like Icarus, he could have said ‘CIRCLE|INURES|RUDEST|CREASE|LESSER|ESTERS’.)]

The conclusion to be drawn about exercises of this kind is that four letters are nothing at all; that five letters are so easy that nothing is worth notice unless the combination have meaning; that six letters, done in any way, are respectable; and that seven letters would be a triumph.

— AUGUSTUS DE MORGAN, in Notes & Queries (3 September 1859)

Henry Dudeney constructed several 7×7 examples and used them in clever puzzles, beginning with ‘PALATED|ANEMONE|LEVANTS|AMASSES|TONSIRE|ENTERER|DESSERT’[The Weekly Dispatch (25 October and 8 November 1896)] and ‘BOASTER|OBSCENE|ASSERTS|SCEPTRE|TERTIAN|ENTRANT|RESENTS’ [The Weekly Dispatch (21 November and 5 December 1897)]. Years later he was particularly pleased to have found ‘NESTLES|ENTRANT| STRANGE|TRAITOR|LANTERN|ENGORGE|STERNER’ [Strand 55 (1918), 488; 56 (1919), 74; The World’s Best Word Puzzles (1925), Puzzles 142 and 145]. M. Douglas McIlroy was the first to apply computers to this task [Word Ways 8 (1975), 195–197], discovering 52 examples such as ‘WRESTLE|RENEWAL|ENPLANE|SELFDOM|TWADDLE|LANOLIN|ELEMENT’. Then he turned to the more difficult problem of double word squares, which are un-symmetric and contain 2n distinct words: He presented 117 double squares, such as ‘REPAST|AVESTA|CIRCUS|INSECT|SCONCE|MENTOR’, in Word Ways 9 (1976), 80–84. (His experiments allowed proper names, but avoided plurals and other derived word forms.)

For an excellent history of word squares and word cubes, chronicling the subsequent computer developments as well as extensive searches for 10 × 10 examples using vast dictionaries, see Ross Eckler, Making the Alphabet Dance (New York: St. Martin’s Griffin, 1997), 188–203; Tribute to a Mathemagician (A K Peters, 2005), 85–91.

33. Working from bottom to top and right to left is equivalent to working from top to bottom and left to right on the word reversals. This idea does make the tries smaller; but unfortunately it makes the programs slower. For example, the 6×5 computation of answer 28 involves a 6347-node trie for the 6-letter words and a 63060-node compressed trie for the 5-letter words. Those sizes go down to 5188 and 56064, respectively, when we reverse the words; but the running time goes up from 380 Gμ to 825 Gμ.

34. Leave out face and (of course) dada; the remaining eleven are fine.

35. Keep tables pi, pij′, pijk", si, sij′, sijk", for 0 ≤ i, j, k < m, each capable of storing a ternary digit, and initially zero. Also keep a table x_0, x_1, ... of tentatively accepted words. Begin with g ← 0. Then for each input wj = abcd, where 0 ≤ a, b, c, d < m, set xg ← abcd and also do the following: Set pa←pa+˙1, pab′←pab′+˙1, pabc"←pabc"+˙1, sd←sd+˙1, scd′←scd′+˙1, sbcd"←sbcd"+˙1, where x+˙y=min(2,x+y)) denotes saturating ternary addition. Then if sa′pb′c′d′"+sa′b′"pd′=0 for all xk = a′b′c′d′, where 0 ≤ k ≤ g, set g ← g +1. Otherwise reject wj and set pa ← pa −1, pab′←pab′−1, pabc"←pabc"−1, sd ← sd − 1, scd′←scd′−1, sbcd"←sbcd"−1.

36. (a) The word bc appears in message abcd if and only if a → b, b → c, and c → d.

(b) For 0 ≤ k < r, put vertex v into class k if the longest path from v has length k. Given any such partition, we can include all arcs from class k to class j < k without increasing the path lengths. So it’s a question of finding the maximum of Σ0≤j<k<rpjpk subject to p_0 +p_1 + ... +pr−1 = m. The values pj = ⌊(m+j)/r⌋ achieve this (see exercise 7.2.1.4–68(a)). When r = 3 the maximum simplifies to ⌊m^2/3 ⌋.

37. (a) The factors of the period, 15 926 535 89 79 323 8314, begin at the respective boundary points 3, 5, 8, 11, 13, 15, 18 (and then 3 + 19 = 22, etc.). Thus round 1 retains boundaries 5, 8, and 15. The second-round substrings y_0 = 926, y_1 = 5358979, y_2 = 323831415 have different lengths, so lexicographic comparison is unnecessary; the answer is y_2y_0y_1 = x_15 ...x_33.

(b) Each substring consists of at least three substrings of the previous round.

(c) Let a_0 = 0, b_0 = 1, ae+1 = aeaebe, be+1 = beaebe; use ae or be when n = 3e.

(d) We use an auxiliary subroutine ‘less(i)’, which returns [yi−1 <yi], given i > 0: If bi − bi−1 ≠ bi+1 − bi, return [bi − bi−1 <bi+1 − bi]. Otherwise, for j = 0, 1, ... , while bi + j < bi+1, if xbi−1+j ≠ xbi+j return [xbi−1+j <xbi+j]. Otherwise return 0.

The tricky part of the algorithm is to discard initial factors that aren’t periodic. The secret is to let i_0 be the smallest index such that yi−3 ≥ yi−2 < yi−1; then we can be sure that a factor begins with yi.

O1. [Initialize.] Set xj ← xj−n for n ≤ j < 2n, bj ← j for 0 ≤ j < 2n, and t ← n.

O2. [Begin a round.] Set t′ ← 0. Find the smallest i > 0 such that less(i) = 0. Then find the smallest j ≥ i + 2 such that less(j − 1) = 1 and j ≤ t + 2. (If no such j exists, report an error: The input x was equal to one of its cyclic shifts.) Set i ← i_0 ← j mod t. (Now a dip of the period begins at i_0.)

O3. [Find the next factor.] Find the smallest j ≥ i + 2 such that less(j − 1) = 1. If j − i is even, go to O5.

O4. [Retain a boundary.] If i < t, set bt′′←bi; otherwise set bk′←bk−1′ for t′ ≥ k > 0 and b0′←bi−t. Finally set t′ ← t′ +1.

O5. [Done with round?] If j < i_0 + t, set i ← j and return to O3. Otherwise, if t′ = 1, terminate; σx begins at item xb0′. Otherwise set t ← t′, bk←bk′ for 0 ≤ k < t, and bk ← bk−t + n for k ≥ t while bk−t < 2n. Return to O2.

(e) Say that a “superdip” is a dip of odd length followed by zero or more dips of even length. Any infinite sequence y that begins with an odd-length dip has a unique factorization into superdips. Those superdips can, in turn, be regarded as atomic elements of a higher-level string that can be factored into dips. The result σx of Algorithm O is an infinite periodic sequence that allows repeated factorization into infinite periodic sequences of superdips at higher and higher levels, until becoming constant.

Notice that the first dip of σx ends at position i_0 in the algorithm, because its length isn’t 2. Therefore we can prove the commafree property by observing that, if codeword σx″ appears within the concatenation σx σx′ of two codewords, its superdip factors are also superdip factors of those codewords. This yields a contradiction if any of σx, σx′,or σx″ is a superdip. Otherwise the same observation applies to the superdip factors at the next level. [Eastman’s original algorithm was essentially the same, but presented in a more complicated way; see IEEE Trans. IT-11 (1965), 263–267. R. A. Scholtz subsequently discovered an interesting and totally different way to define the set of codewords produced by Algorithm O, in IEEE Trans. IT-15 (1969), 300–306.]

38. Let fk(m) be the number of dips of length k for which m > z_1 and zk < m. The number of such sequences with zk−1 = j is (m−j−1) (m−j+k−3k−2)=(k−1)(m−j+k−3k−1); summing for 0 ≤ j < m gives fk(m)=(k−1)(m+k−2k). Thus Fm(z)=Σk=0∞⁢ fk(m)zk=(mz−1)/(1−z)m. (The fact that f_0(m) = −1 in these formulas turns out to be useful!)

Algorithm O finishes in one round if and only if some cyclic shift of x is a superdip. The number of aperiodic x that finish in one round is therefore n[zn] Gm(z), where

Gm(z)=Fm(−z)−Fm(z)Fm(−z)+Fm(z)=(1+mz)(1−z)m−(1−mz)(1+z)m(1+mz)(1−z)m+(1−mz)(1+z)m.

To get the stated probability, divide by Σd\n μ(d)mn/d, the number of aperiodic x. (See Eq. 7.2.1.1–(60). For n = 3, 5, 7, 9 these probabilities are 1, 1, 1, and 1−3/(m3−13).)

39. If so, it couldn’t have 0011, 0110, 1100, or 1001.

40. That section considered such representations of stacks and queues, but not of unordered sets, because large blocks of sequential memory were either nonexistent or ultra-expensive in olden days. Linked lists were the only decent option for families of variable-size sets, because they could more readily fit in a limited high-speed memory.

41. (a) The blue word x with α = d (namely 1101) appears in its P2 list at location 5e.

(b) The P3 list for words of the form 010∗ is empty. (Both 0100 and 0101 are red.)

42. (a) The S2 list of 0010 has become closed (hence 0110 and 1110 are hidden).

(b) Word 1101 moved to the former position of 1001 in its S1 list, when 1001 became red. (Previously 1011 had moved to the former position of 0001.)

43. In this case, which of course happens rarely, it’s safe to set all elements of STAMP to zero and set σ ← 1. (Do not be tempted to save one line of code by setting all STAMP elements to −1 and leaving σ = 0. That might fail when σ reaches the value −1!)

44. (a) Set r ← 5. Then for k ← 0, 1, ... , f − 1, set t ← FREE[k], j ← MEM[CLOFF + 4t + m^4] − (CLOFF +4t), and if j < r set r ← j, c ← t; break out of the loop if r = 0.

(b) If r > 0 set x ← MEM[CLOFF +4cl(ALF[x])].

(c) If r > 1 set q ← 0, p′ ← MEM[PP], and p ← POISON. While p < p′ do the following steps: Set y ← MEM[p], z ← MEM[p +1], y′ ← MEM[y + m^4], and z′ ← MEM[z + m^4]. (Here y and z point to the heads of prefix or suffix lists; y′ and z′ point to the tails.) If y = y′ or z = z′, delete entry p from the poison list; this means, as in (18), to set p′ ← p′−2, and if p ≠ p′ to store(p, MEM[p′]) and store(p+1, MEM[p′ +1]). Otherwise set p ← p+2; if y′ −y ≥ z′ −z and y′ −y > q, set q ← y′ −y and x ← MEM[z]; if y′ − y < z′ − z and z′ − z > q, set q ← z′ − z and x ← MEM[y]. Finally, after p has become equal to p′, store( PP,p′) and set c ← cl( ALF[x]). (Experiments show that this “max kill” strategy for r > 1 slightly outperforms a selection strategy based on r alone.)

45. (a) First there’s a routine rem(α, δ, o) that removes an item from a list, following the protocol (21): Set p ← δ + o and q ← MEM[p + m^4] − 1. If q ≥ p (meaning that list p isn’t closed or being killed), store(p + m^4,q), set t ← MEM[α + o − m^4]; and if t ≠ q also set y ← MEM[q], store(t, y), and store( ALF[y] + o − m^4,t).

Now, to redden x we set α ← ALF[x], store(α, RED); then rem(α, p_1(α), P1OFF), rem(α, p_2(α), P2OFF), ... , rem(α, s_3(α), S3OFF), and rem(α, 4cl(α), CLOFF).

(b) A simple routine close(δ, o) closes list δ+o: Set p ← δ+o and q ← MEM[p + m^4]; if q ≠ p − 1, store(p + m^4,p − 1).

Now, to green x we set α ← ALF[x], store(α, GREEN); then close(p_1(α), P1OFF), close(p_2(α), P2OFF), ... , close(s_3(α), S3OFF), and close(4cl(α), CLOFF). Finally, for p ≤ r < q (using the p and q that were just set within ‘close’), if MEM[r] ≠ x redden MEM[r].

(c) First set p′ ← MEM[PP] +6, and store(p′ −6,p_1(α)+ S1OFF), store(p′ −5,s_3(α)+ P3OFF), store(p′ − 4,p_2(α)+ S2OFF), store(p′ − 3,s_2(α)+ P2OFF), store(p′ − 2,p_3(α)+ S3OFF), store(p′ − 1,s_1(α)+ P1OFF); this adds the three poison items (27).

Then set p ← POISON and do the following while p < p′: Set y, z, y′, z′ as in answer 44(c), and delete poison entry p if y = y′ or z = z′. Otherwise if y′ < y and z′ < z, go to C5 (a poisoned suffix-prefix pair is present). Otherwise if y′ > y and z′ > z, set p ← p + 2. Otherwise if y′ < y and z′ > z, store(z + m^4,z), redden MEM[r ] for z ≤ r < z′, and delete poison entry p. Otherwise (namely if y′ > y and z′ < z), store(y + m^4,y), redden MEM[r] for y ≤ r < y′, and delete poison entry p.

Finally, after p has become equal to p′, store( PP,p′).

46. Exercise 37 exhibits such codes explicitly for all odd n. The earliest papers on the subject gave solutions for n = 2, 4, 6, 8. Yoji Niho subsequently found a code for n = 10 but was unable to resolve the case n = 12 [IEEE Trans. IT-19 (1973), 580–581].

This problem can readily be encoded in CNF and given to a SAT solver. The case n = 10 involves 990 variables and 8.6 million clauses, and is solved by Algorithm 7.2.2.2C in 10.5 gigamems. The case n = 12 involves 4020 variables and 175 million clauses. After being split into seven independent subproblems (by appending mutually exclusive unit clauses), it was proved unsatisfiable by that algorithm after about 86 teramems of computation.

So the answer is “No.” But we can come close: Aaron Windsor used a SAT solver in 2021 to discover a binary commafree code for n = 12 that contains a representative of every cycle class except [000011001011].

47. (a) There are 28 commafree binary codes of size 3 and length 4; Algorithm C produces half of them, because it assumes that cycle class [0001] is represented by 0001 or 0010. They form eight equivalence classes, two of which are symmetric under the operation of complementation-and-reflection; representatives are {0001, 0011, 0111} and {0010, 0011, 1011}. The other six are represented by {0001, 0110, 0111 or 1110}, {0001, 1001, 1011 or 1101}, {0001, 1100, 1101}, {0010, 0011, 1101}.

(b) Algorithm C produces half of the 144 solutions, which form twelve equivalence classes. Eight are represented by {0001, 0002, 1001, 1002, 1102, 2001, 2002, 2011, 2012, 2102, 2112, 2122 or 2212} and ({0102, 1011, 1012} or {2010, 1101, 2101}) and ({1202, 2202, 2111} or {2021, 2022, 1112}); four are represented by {0001, 0020, 0021, 0022, 1001, 1020, 1021, 1022, 1121 or 1211, 1201, 1202, 1221, 2001, 2201, 2202} and ({1011, 1012, 2221} or {1101, 2101, 1222}).

(c) Algorithm C yields half of the 2304 solutions, which form 48 equivalence classes. Twelve classes have unique representatives that omit cycle classes [0123], [0103], [1213], one such being the code {0010, 0020, 0030, 0110, 0112, 0113, 0120, 0121, 0122, 0130, 0131, 0132, 0133, 0210, 0212, 0213, 0220, 0222, 0230, 0310, 0312, 0313, 0320, 0322, 0330, 0332, 0333, 1110, 1112, 1113, 2010, 2030, 2110, 2112, 2113, 2210, 2212, 2213, 2230, 2310, 2312, 2313, 2320, 2322, 2330, 2332, 2333, 3110, 3112, 3113, 3210, 3212, 3213, 3230, 3310, 3312, 3313}. The others each have two representatives that omit classes [0123], [0103], [0121], one such pair being the code {0001, 0002, 0003, 0201, 0203, 1001, 1002, 1003, 1011, 1013, 1021, 1022, 1023, 1031, 1032, 1033, 1201, 1203, 1211, 1213, 1221, 1223, 1231, 1232, 1233, 1311, 1321, 1323, 1331, 2001, 2002, 2003, 2021, 2022, 2023, 2201, 2203, 2221, 2223, 3001, 3002, 3003, 3011, 3013, 3021, 3022, 3023, 3031, 3032, 3033, 3201, 3203, 3221, 3223, 3321, 3323, 3331} and its isomorphic image under reflection and (01)(23).

48. Algorithm C isn’t fast enough to solve this problem. But Aaron Windsor used a SAT solver in 2021 to find such a code of size 139 = (5^4 − 5^2)/4 − 11, and to prove that no such code of size 140 exists. (He also found, rather quickly, that an optimum ternary commafree code for n = 6 contains (3^6 − 3^3 − 3^2 +3^1)/6 − 3 = 113 codewords.)

49. The 3-bit sequences 101, 111, 110 were rejected before seeing 000. In general, to make a uniformly random choice from q possibilities, the text suggests looking at the next t = ⌈lg q⌉ bits b_1 ...bt. If (b_1 ...bt)2 < q, we use choice (b_1 ...bt)2 + 1; otherwise we reject b_1 ...bt and try again. [This simple method is optimum when q ≤ 4, and the best possible running time for other values of q uses more than half as many bits. But a better scheme is available for q = 5, using only 313 bits per choice instead of 445; and for q = 6, one random bit reduces to the case q = 3. See D. E. Knuth and A. C. Yao, Algorithms and Complexity, edited by J. F. Traub (Academic Press, 1976), 357–428, §2.]

50. It’s the number of nodes on level l + 1 (depth l) of the search tree. (Hence we can estimate the profile. Notice that D = D_1 ...Dl−1 in step E2 of Algorithm E.)

51. Z_0 = C(), Zl+1 = c() + D_1 c(X_1)+ D_1D_2c(X_1X_2)+ ... + D_1 ...Dl c(X_1 ...Xl)+ D_1 ...Dl+1C(X_1 ...Xl+1).

52. (a) True: The generating function is z(z +1) ... (z + n − 1)/n!; see Eq. 1.2.10–(9).

(b) For instance, suppose Y_1Y_2 ...Yl = 1457 and n = 9. Alice’s probability is 111223141556177889=13141619. Elmo obtains X_1X_2 ...Xl = 7541 with probability 19161413.

(c) The upper tail inequality (see exercise 1.2.10–22 with μ = Hn) tells us that Pr(l ≥ (ln n)(ln ln n)) ≤ exp(−(ln n)(ln ln n)(ln ln ln n)+ O(ln n)(ln ln n)).

(d) If k ≤ n/3 we have Σj=0k(nj)≤2(nk). By exercise 1.2.6–67, the number of nodes on the first (ln n)(ln ln n) levels is therefore at most 2(ne/((ln n)(ln ln n)))(ln n)(ln ln n).

53. The key idea is to introduce recursive formulas analogous to (29):

m(x1⁢ … xl)=c(x1…xl)+min(m(x1…xlxl+1(1)) d,…,m(x1…xlxl+1(d))d);M(x1⁢ …⁢ xl)=c(x1…xl)+max(M(x1…xlxl+1(1)) d,…,M(x1…xlxl+1(d))d);C^(x1⁢…⁢ xl)=c(x1…xl)2+Σi=1d(C^(x1…xlxl+1(i))d+2c(x1…xl)C(x1…xlxl+1(i))).

They can be computed via auxiliary arrays MIN, MAX, KIDS, COST, and CHAT as follows:

At the beginning of step B2, set MIN[l] ← ∞, MAX[l] ← KIDS[l] ← COST[l] ← CHAT[l] ← 0. Set KIDS[l] ← KIDS[l] + 1 just before l ← l + 1 in step B3.

At the beginning of step B5, set m ← c(x_1 ...xl−1) + KIDS[l] × MIN[l], M ← c(x_1 ...xl−1) + KIDS[l] × MAX[l], C ← c(x_1 ...xl−1) + COST[l], Ĉ ← c(x_1 ...xl−1)2 + KIDS[l] × CHAT[l] + 2 × COST[l]. Then, after l ← l − 1 is positive, set MIN[l] ← min(m, MIN[l]), MAX[l] ← max(M, MAX[l]), COST[l] ← COST[l] + C, CHAT[l] ← CHAT[l] + Ĉ. But when l reaches zero in step B5, return the values m, M, C, Ĉ − C^2.

54. Let p(i) = pX_1 ... Xl−1(yi), and simply set D ← D/p(I) instead of D ← Dd. Then node x_1 ...xl is reached with probability Π(x_1 ...xl) = p(x_1)px_1(x_2) ... px_1 ... xl(xl), and c(x_1 ... xl) has weight 1/Π(x_1 ...xl) in S; the proof of Theorem E goes through as before. Notice that p(I) is the a posteriori probability of having taken branch I.

(The formulas of answer 53 should now use ‘/p(i)’ instead of ‘d’; and that algorithm should be modified appropriately, no longer needing the KIDS array.)

55. Let pX_1...Xl−1 (yi) = C(x_1 ...xl−1yi)/(C(x_1 ...xl−1) − c(x_1 ...xl−1)). (Of course we generally need to know the cost of the tree before we know the exact values of these ideal probabilities, so we cannot achieve zero variance in practice. But the form of this solution shows what kinds of bias are likely to reduce the variance.)

56. The effects of lookahead, dynamic ordering, and reversible memory are all captured easily by a well-designed cost function at each node. But there’s a fundamental difference in step C2, because different codeword classes can be selected for branching at the same node (that is, with the same ancestors x_1 ...xl−1) after C5 has undone the effects of a prior choice. The level l never surpasses L + 1, but in fact the search tree involves hidden levels of branching that are implicitly combined into single nodes.

Thus it’s best to view Algorithm C’s search tree as a sequence of binary branches: Should x be one of the codewords or not? (At least this is true when the “max kill” strategy of answer 44 has selected the branching variable x. But if r > 1 and the poison list is empty, an r-way branch is reasonable (or an (r + 1)-way branch when the slack is positive), because r will be reduced by 1 and the same class c will be chosen after x has been explored.)

If x has been selected because it kills many other potential codewords, we probably should bias the branch probability as in exercise 54, giving smaller weight to the “yes” branch because the branch that includes x is less likely to lead to a large subtree.

57. Let pk = 1/D(k) be the probability that Algorithm E terminates at the kth leaf. Then Σk=1M(1/M)lg(1/(Mpk)) is the Kullback–Leibler divergence D(q||p), where q is the uniform distribution (see exercise MPR–121). Hence 1M⁢  Σk=1M⁢ lg D(k)≥lg M. (The result of this exercise is essentially true in any probability distribution.)

58. Let ∞ be any convenient value ≥ n. When vertex v becomes part of the path we will perform a two-phase algorithm. The first phase identifies all “tarnished” vertices, whose DIST must change; these are the vertices u from which every path to t passes through v. It also forms a queue of “resource” vertices, which are untarnished but adjacent to tarnished ones. The second phase updates the DISTs of all tarnished vertices that are still connected to t. Each vertex has LINK and STAMP fields in addition to DIST.

For the first phase, set d ← DIST(v ), DIST(v ) ← ∞+1, R ← Λ, T ← v, LINK(v ) ← Λ, then do the following while T ≠ Λ: (∗) Set u ← T, T ← S ← Λ. For each w −−− u, if DIST(w ) < d do nothing (this happens only when u = v); if DIST(w ) ≥ ∞ do nothing (w is gone or already known to be tarnished); if DIST(w ) = d, make w a resource (see below); otherwise DIST(w ) = d +1. If w has no neighbor at distance d, w is tarnished: Set LINK(w ) ← T, DIST(w ) ← ∞, T ← w. Otherwise make w a resource (see below). Then set u ← LINK(u ), and return to (∗) if u ≠ Λ.

The queue of resources will start at R. We will stamp each resource with v so that nothing is added twice to that queue. To make w a resource when DIST(w ) = d, do the following (unless u = v or STAMP(w ) = v): Set STAMP(w ) ← v; if R = Λ, set R ← RT ← w; otherwise set LINK(RT) ← w and RT ← w. To make w a resource when DIST(w ) = d + 1 and u ≠ v and STAMP(w ) ≠ v, put it first on stack S as follows: Set STAMP(w ) ← v; if S = Λ, set S ← SB ← w; otherwise set LINK(w ) ← S, S ← w.

Finally, when u = Λ, we append S to R: Nothing needs to be done if S = Λ. Otherwise, if R = Λ, set R ← S and RT ← SB; but if R ≠ Λ, set LINK(RT) ← S and RT ← SB. (These shenanigans keep the resource queue in order by DIST.)

Phase 2 operates as follows: Nothing needs to be done if R = Λ. Otherwise we set LINK(RT) ← Λ, S ← Λ, and do the following while R ≠ Λ or S ≠ Λ: (i) If S = Λ, set d ← DIST(R). Otherwise set u ← S, d ← DIST(u ), S ← Λ; while u ≠ Λ, update the neighbors of u and set u ← LINK(u ). (ii) While R ≠ Λ and DIST(R) = d, set u ← R, R ← LINK(u ), and update the neighbors of u. In both cases “update the neighbors of u” means to look at all w −−− u, and if DIST(w ) = ∞ to set DIST(w ) ← d + 1, STAMP(w ) ← v, LINK(w ) ← S, and S ← w. (It works!)

59. (a) Compute the generating function g(z) (see exercise 7.1.4–209) and then g′(1).

(b) Let (A, B, C) denote paths that touch (center, NE corner, SW corner). Recursively compute eight counts (c_0,...,c_7) at each node, where cj counts paths π with j = 4[π ∈ A] + 2[π ∈ B] + [π ∈ C ]. At the sink node we have c_0 = 1, c_1 = ... = c_7 = 0. Other nodes have the form x = (ē? xl: xh) where e is an edge. Two edges go across the center and affect A; three edges affect each of B and C. Say that those edges have types 4, 2, 1, respectively; other edges have type 0. Suppose the counts for xl and xh are (c0′,…,c7′) and (c0",…,c7"), and e has type t. Then count cj for node x is cj′+[t=0]cj"+[t&j≠0](cj"+cj−t").

(This procedure yields the following exact “Venn diagram” set counts at the root: c0|=|A¯∩B¯∩C¯|=7653685384889019648091604; c1=c2=|A¯∩B¯∩C|=|A¯∩B∩C¯|=7755019053779199171839134; c3=|A¯∩B∩C|=|7857706970503366819944024; c4=|A∩B¯∩C¯|=4888524166534573765995071; c5=c6=|A∩B¯∩C|=A∩   B∩C¯|=4949318991771252110605148; c7=|A∩B∩C|=5010950157283718807987280.)

60. Yes, the paths are less chaotic and the estimates are better:

61. (a) Let xk be the number of nodes at distance k − 1 from the root.

(b) Let Qn(m)=Pn(1)+…+Pn(m). Then we have the joint recurrence P1(m)=1, Pn+1(m)=Qn(2m); in particular, Q1(m)=m. And for n ≥ 2, we have Qn⁢(m)=Σk=1n⁢ ank(mk) for certain constants ank that can be computed as follows: Set tk`←Pn(k) for 1 ≤ k ≤ n. Then for k = 2, ... , n set tn ← tn − tn−1, ... , tk ← tk − tk−1. Finally ank ← tk for 1 ≤ k ≤ n. For example, a_21 = a_22 = 2; a_31 = 6, a_32 = 14, a_33 = 8. The numbers Pn(m) have O(n^2 + n log m) bits, so this method needs O(n^5) bit operations to compute Pn.

(c) Pn(m) corresponds to random paths with X_1 = m, Dk = 2Xk, Xk+1 = ⌈2UkXk⌉, where each Uk is an independent uniform deviate. Therefore Pn(m)=E(D1⁢ … Dn−1) is the number of nodes on level n of an infinite tree. We have Xk+1 ≥ 2kU_1 ...Ukm, by induction; hence   Pn(m)≥E(2(n2)U1n−2U2n−3⁢  … Un−2mn−11)=2(n3)mn−1/(n−1)! .

[M. Cook and M. Kleber have discussed similar sequences in Electronic Journal of Combinatorics 7 (2000), #R44, 1–16. See also K. Mahler’s asymptotic formula for binary partitions, in J. London Math. Society 15 (1940), 115–123, which shows that lg Pn=(n2)−lg(n−1)!+(lg n2)+O(1).]

62. Random trials indicate that the expected number of 2-regular graphs is ≈ 3.115, and that the number of disjoint pairs is (0, 1, ... , 9, and ≥ 10) approximately (74.4, 4.3, 8.7, 1.3, 6.2, 0.2, 1.5, 0.1, 2.0, 0.0, and 12.2) percent of the time. If the cubes are restricted to cases where each color occurs at least five times, these numbers change to ≈ 4.89 and (37.3, 6.6, 17.5, 4.1, 16.3, 0.9, 5.3, 0.3, 6.7, 0.2, 5.0).

However, the concept of “unique solution” is tricky, because a 2-regular graph with k cycles yields 2k ways to position the cubes. Let’s say that a set of cubes has a strongly unique solution if (i) it has a unique disjoint pair of 2-regular graphs, and furthermore (ii) both elements of that pair are n-cycles. Such sets occur with probability only about 0.3% in the first case, and 0.4% in the second.

[N. T. Gridgeman, in Mathematics Magazine 44 (1971), 243–252, showed that puzzles with four cubes and four colors have exactly 434 “types” of solutions.]

63. It’s easy to find such examples at random, as in the second part of the previous answer, since strongly unique sets occur about 0.5% of the time (and weakly unique sets occur with probability ≈ 8.4%). For example, the pairs of opposite faces might be (12, 13, 34), (02, 03, 14), (01, 14, 24), (04, 13, 23), (01, 12, 34).

(Incidentally, if we require each color to occur exactly six times, every set of cubes that has at least one solution will have at least three solutions, because the “hidden” pairs can be chosen in three ways.)

64. Each of these cubes can be placed in 16 different ways that contribute legitimate letters to all four of the visible words. (A cube whose faces contain only letters in {C, H, I, N, O, U, X, Z} can be placed in 24 ways. A cube with a pattern like cannot be placed at all.) We can restrict the first cube to just two placements; thus there are 2 · 16 · 16 · 16 · 16 = 131072 ways to place those cubes without changing their order. Of these, only 6144 are “compatible,” in the sense that no right-side-up-only letter appears together with an upside-down-only letter in the same word.

The 6144 compatible placements can then each be reordered in 5! = 120 ways. One of them, whose words before reordering are GRHTI, NCICY, , OUNNI, leads to the unique solution. (There’s a partial solution with three words out of four. There also are 39 ways to get two valid words, including one that has UNTIL adjacent to HOURS, and several with SYRUP opposite ECHOS.)

65. E. Robertson and I. Munro, in Utilitas Mathematica 13 (1978), 99–116, have reduced the exact cover problem to this problem.

66. Call the rays N, NE, E, SE, S, SW, W, NW; call the disks 1, 2, 3, 4 from inside to outside. We can keep disk 1 fixed. The sum of rays N, S, E, W must be 48. It is 16 (on disk 1) plus 13 or 10 (on disk 2) plus 8 or 13 (on disk 3) plus 11 or 14. So it is attained either as shown, or after rotating disks 2 and 4 clockwise by 45°. (Or we could rotate any disk by a multiple of 90°, since that keeps the desired sum invariant.)

Next, with optional 90° rotations, we must make the sum of rays N + S equal to 24. In the first solution above it is 9 plus (6 or 7) plus (4 or 4) plus (7 or 4), hence never 24. But in the other solution it’s 9 plus (4 or 6) plus (4 or 4) plus (5 or 9); hence we must rotate disk 2 clockwise by 90°, and possibly also disk 3. However, 90° rotation of disk 3 would make the NE + SW sum equal to 25, so we musn’t move it.

Finally, to get NE’s sum to be 12, via optional rotations by 180°, we have 1 plus (2 or 5) plus (1 or 5) plus (3 or 4); we must shift disks 3 and 4. Hurrah: That makes all eight rays correct. Factoring twice has reduced 8^3 trials to 2^3 +2^3 +2^3.

[See George W. Ernst and Michael M. Goldstein, JACM 29 (1982), 1–23. Such puzzles go back to the 1800s; three early examples are illustrated on pages 28 of Slocum and Botermans’s New Book of Puzzles (1992). One of them, with six rings and six rays, factors from 6^5 trials to 2^5 +3^5. A five-ray puzzle would have defeated factorization.]

67. Call the cards 1525, 5113, ... , 3755. The key observation is that all 12 sums must be odd, so we can first solve the problem mod 2. For this purpose we may call the cards 1101, 1111, ... , 1111; only three cards now change under rotation, namely 1101, 0100, and 1100 (which are the mod 2 images of 1525, 4542, and 7384).

A second observation is that each solution gives 6 × 6 × 2 others, by permuting rows and/or columns and/or by rotating all nine cards. Hence we can assume that the upper left card is 0011 (8473). Then 0100 (4542) must be in the first column, possibly rotated to 0001 (4245), to preserve parity in the left two black sums. We can assume that it’s in row 2. In fact, after retreating from 13 mod 2 to 13, we see that it must be rotated. Hence the bottom left card must be either 4725, 7755, or 3755.

Similarly we see that 1101 (1525) must be in the first row, possibly rotated to 0111 (2515); we can put it in column 2. It must be rotated, and the top right card must be 3454 or 3755. This leaves just six scenarios to consider, and we soon obtain the solution: 8473, 2515, 3454; 4245, 2547, 7452; 7755, 1351, 5537.

68. In general, let’s say that a vertex labeling of a digraph is stable if v’s label is the number of distinct labels among {w | v −−→ w}, for all v. We wish to find all stable labelings that extend a given partial labeling. We may assume that no vertex is a sink.

Let Λ(v) be a set of digits that includes every label that v could possibly have, in a solution to this extension problem. Initially, Λ(v) = {d} if v’s label is supposed to be d; otherwise Λ(v) = {1,...,d+(v)}. These sets are conveniently represented as the binary numbers L(v) = {2k–1 | k ∈ Λ(v)}. Our goal is to reduce each L(v) to a 1-bit number. A nice backtrack routine called “refine(v)” proves to be helpful in this regard.

Let v_0 = v and let v_1, ... , vn be v’fs successors. Let aj = L(vj). Following the outline of Algorithm B, we let xl ⊆ al be a 1-bit number, accepted in step B3 only if 2νsl–1 ⊆ gl, where sl = x_1 | ... | xl and where the goal sets gl are defined by gn = a_0, gl = (gl+1 | gl+1 ≫ 1) & (2l – 1). We start with all bj → 0; then when visiting a solution x_1 . . . xn, we set bj → bj | xj for 1 ≤ j ≤ n, and b_0 ≠ b_0 | 2νsn–1. After finding all solutions we’ll have bj ⊆ aj for all j; and whenever bj ≠ aj we can reduce L(vj) → bj.

Operate in rounds, where all vertices are refined in round 1; subsequent rounds refine only the vertices whose parameters aj have changed. In each round we first refine the vertices with smallest product (νa_1) ... (νan), because they have the fewest potential solutions x_1 ... xn. This method isn’t guaranteed to succeed; but fortunately it does solve the stated problem, after 301 refinements in 6 rounds. [Such “Japanese arrow puzzles” were introduced by Masanori Natsuhara on page 75 of Puzuraa 128 (July 1992).]

69. (The 33rd boxed clue will, of course, have to point outside the 10 × 10 array. Maybe there’s even a puzzle whose empty boxes are symmetrical, as in exercise 68.)

70. An extremely instructive analysis [Combinatorics, Probability and Computing 23 (2014), 725–748] leads to the recurrences Pm = (5 + 9z)Pm−2 − 4Pm−4, Qm = (5 + 9z)Qm−2 − 4Qm−4, for m ≥ 6, where the initial values are (P_2,P_3,P_4,P_5) = (1, 1+ z, 1+3z, 1+10z +9z^2); (Q_2,Q_3,Q_4,Q_5) = (1 − 4z, 1 − 9z − 6z^2, 1 − 19z − 18z^2, 1 − 36z − 99z^2 − 54z^3). The denominator Qm(z) has all real roots, exactly one of which is positive, namely 1/ρm.

71. Suppose there are n questions, whose answers each lie in a given set S. A student supplies an answer list α = a_1 ...an, with each aj ∈ S; a grader supplies a Boolean vector β = x_1 ...xn. There is a Boolean function fjs(α, β) for each j ∈ {1,...,n} and each s ∈ S. A graded answer list (α, β) is valid if and only if F (α, β) is true, where

F(α,β)=F(a1…an,x1…xn)=Λj=1nΛs∈S([aj=s]⇒xj≡fjs(α,β)).

The maximum score is the largest value of x_1 + ··· + xn over all graded answer lists (α, β) that are valid. A perfect score is achieved if and only if F (α, 1 ... 1) holds.

Thus, in the warmup problem we have n = 2, S = {A, B}; f1A = [a_2 =B]; f1B = [a_1 =A]; f2A = x_1; f2B=x¯2⊕[a1=A].. The four possible answer lists are:

AA:F=(x1≡[A=B])∧(x2≡x1)AB:F=(x1≡[B=B])∧(x2≡x¯2⁢ ⊕ [A=A])BA:F=(x1≡[B=A])∧(x2≡x1)BB:F=(x1≡[B=A])∧(x2≡x¯2⁢ ⊕ [B=A])

Thus AA and BA must be graded 00; AB can be graded either 10 or 11; and BB has no valid grading. Only AB can achieve the maximum score, 2; but 2 isn’t guaranteed.

In Table 666 we have, for example, f1C = [a_2 ≠ = A] ∧ [a_3 = A]; f4D = [a_1 = D] ∧ [a_15 = D]; f12A = [ΣA − 1 = ΣB], where Σs = Σ1≤j≤20[aj = s]. It’s amusing to note that f14E = [{ΣA,..., ΣE} = {2, 3, 4, 5, 6}].

The other cases are similar (although often more complicated) Boolean functions — except for 20D and 20E, which are discussed further in exercise 72.

Notice that an answer list that contains both 10E and 17E must be discarded: It can’t be graded, because 10E says ‘x10≡x¯17’ while 17E says ‘x_17 ≡ x_10’.

By suitable backtrack programming, we can prove first that no perfect score is possible. Indeed, if we consider the answers in the order (3, 15, 20, 19, 2, 1, 17, 10, 5, 4, 16, 11, 13, 14, 7, 18, 6, 8, 12, 9), many cases can quickly be ruled out. For example, suppose a_3 = C. Then we must have a_1 ≠ a_2 ≠ ··· = a_16 ≠ a_17 = a_18 ≠ a_19 ≠ a_20, and early cutoffs are often possible. (We might reach a node where the remaining choices for answers 5, 6, 7, 8, 9 are respectively {C, D}, {A, C}, {B, D}, {A, B, E}, {B, C, D}, say. Then if answer 8 is forced to be B, answer 7 can only be D; hence answer 6 is also forced to be A. Also answer 9 can no longer be B.) An instructive little propagation algorithm will make such deductions nicely at every node of the search tree. On the other hand, difficult questions like 7, 8, 9, are best not handled with complicated mechanisms; it’s better just to wait until all twenty answers have been tentatively selected, and to check such hard cases only when the checking is easy and fast. In this way the author’s program showed the impossibility of a perfect score by exploring just 52859 nodes, after only 3.4 megamems of computation.

The next task was to try for score 19 by asserting that only xj is false. This turned out to be impossible for 1 ≤ j ≤ 18, based on very little computation whatsoever (especially, of course, when j = 6). The hardest case, j = 15, needed just 56 nodes and fewer than 5 kilomems. But then, ta da, three solutions were found: One for j = 19 (185 kilonodes, 11 megamems) and two for j = 20 (131 kilonodes, 8 megamems), namely

1234567891011121314151617181920DCEABEBCEABEAEDBDAbB(i)AEDCABCDCACEDBCADAAc(ii)DCEABADCDAEDAEDBDBBe(iii)

(The incorrect answers are shown here as lowercase letters. The first two solutions establish the truth of 20B and the falsity of 20E.)

72. Now there’s only one list of answers with score ≥ 19, namely (iii). But that is paradoxical — because it claims 20E is false; hence the maximum score cannot be 19!

Paradoxical situations are indeed possible when the global function F of answer 71 is used recursively within one or more of the local functions fjs. Let’s explore a bit of recursive territory by considering the following two-question, two-letter example:

1. (A) Answer 1 is incorrect.

(B) Answer 2 is incorrect.

2. (A) Some answers can’t be graded consistently.

(B) No answers achieve a perfect score.

Here we have f1A=x¯1; f1B=x¯2; f2A = ∃a_1 ∃a_2 ∀x_1 ∀x_2¬F(a_1a_2, x_1x_2); f2B = ∀a_1 ∀a_2¬F(a_1a_2, 11). (Formulas quantified by ∃a or ∀a expand into |S| terms, while ∃x or ∀x expand into two; for example, ∃a∀xg(a, x) = (g(A, 0)∧g(A, 1))∨(g(B, 0)∧g(B, 1)) when S = {A, B}.) Sometimes the expansion is undefined, because it has more than one “fixed point”; but in this case there’s no problem because f2A is true: Answer AA can’t be graded, since 1A implies x1≡x¯1. Also f2B is true, because both BA and BB imply x1≡x¯2. Thus we get the maximum score 1 with either BA or BB and grades 01.

On the other hand the simple one-question, one-letter questionnaire ‘1. (A) The maximum score is 1’ has an indeterminate maximum score. For in this case f1A = F (A, 1). We find that if F (A, 1) = 0, only (A, 0) is a valid grading, so the only possible score is 0; similarly, if F (A, 1) = 1, the only possible score is 1.

OK, suppose that the maximum score for the modified Table 666 is m. We know that m < 19; hence (iii) isn’t a valid grading. It follows that 20E is true, which means that every valid graded list of score m has x_20 false. And we can conclude that m = 18, because of the following two solutions (which are the only possibilities with 20C false):

1234567891011121314151617181920BAdABEDCDAEDAEDEDBEcAEDCABCDCACEDBaCDAAc

But wait: If m = 18, we can score 18 with 20A true and two errors, using (say)

1234567891011121314151617181920DeDABEDeCAEDAEDBDCCA
or 47 other answer lists. This contradicts m = 18, because x_20 is true.

End of story? No. This argument has implicitly been predicated on the assumption that 20D is false. What if m is indeterminate? Then a new solution arises

1234567891011121314151617181920DCEABEDCEAEBAEDBDAdD
of score 19. With (iii) it yields m = 19! If m is determinate, we’ve shown that m cannot actually be defined consistently; but if m is indeterminate, it’s definitely 19.

Question 20 was designed to create difficulties. [:-)]

— DONALD R. WOODS (2001)

73. The 29 words spark, often, lucky, other, month, ought, names, water, games, offer, lying, opens, magic, brick, lamps, empty, organ, noise, after, raise, drink, draft, backs, among, under, match, earth, roofs, topic yield this: “The success or failure of backtrack often depends on the skill and ingenuity of the programmer. ... Backtrack programming (as many other types of programming) is somewhat of an art.” — Solomon W. Golomb, Leonard D. Baumert.

That solution can be found interactively, using inspired guesses based on a knowledge of English and its common two-letter and three-letter words. But could a computer that knows common English words discover it without understanding their meanings?

We can formulate that question as follows: Let w_1, ... , w_29 be the unknown words from WORDS(1000), and let q_1, ... , q_29 be the unknown words of the quotation. (By coincidence there happen to be just 29 of each.) We can restrict the q’s to words that appear, say, 32 times or more in the British National Corpus. That gives respectively (85, 562, 1863, 3199, 4650, 5631, 5417, 4724, 3657, 2448) choices for words of (2, 3, ..., 11) letters; in particular, we allow 3199 possibilities for the five-letter words q_7, q_11, q_21, q_22, because they aren’t required to lie in WORDS(1000). Is there a unique combination of words wi and qj that meets the given anacrostic constraints?

This is a challenging problem, whose answer turns out (surprisingly?) to be no. In fact, here is the first solution found by the author’s machine(!): “The success or failure of backtrack often depends on roe skill and ingenuity at the programmer. ... Backtrack programming (as lacy offal types of programming) as somewhat al an art.” (The OSPD4 includes ‘al’ as the name of the Indian mulberry tree; the BNC has ‘al’ 3515 times, mostly in unsuitable contexts, but that corpus is a blunt instrument.) Altogether 720 solutions satisfy the stated constraints; they differ from the “truth” only in words of at most five letters.

Anacrostic puzzles, which are also known by other names such as double-crostics, were invented in 1933 by E. S. Kingsley. See E. S. Spiegelthal, Proceedings of the Eastern Joint Computer Conference 18 (1960), 39–56, for an interesting early attempt to solve them — without backtracking — on an IBM 704 computer.

74. Instead of considering 1000 possibilities for 131⁢ 132⁢ 133 134 135, it suffices to consider the 43 pairs xy such that cxyab is in WORDS(1000) and abc is a common three-letter word. (Of these pairs ab, ag, ... , ve, only ar leads to a solution. And indeed, the 720 solutions factor into three sets of 240, corresponding to choosing earth, harsh, or large as the keyword for 131⁢ 132⁢ 133 134 135.) Similar reductions, but not so dramatic, occur with respect to 137⁢ 139,⁢ 118 119, 46⁢ 48, and 32⁢ 35.

75. The following algorithm uses an integer utility field TAG(u ) in the representation of each vertex u, representing the number of times u has been “tagged.” The operations “tag u” and “untag u” stand respectively for TAG(u ) ← TAG(u ) + 1 and TAG(u ) ← TAG(u )−1. Vertices shown as ‘⊙’ in the 21 examples have a nonzero TAG field, indicating that the algorithm has decided not to include them in this particular H.

State variables vl (a vertex), il (an index), and al (an arc) are used at level l for 0 ≤ l < n. We assume that n > 1.

R1. [Initialize.] Set TAG(u ) ← 0 for all vertices u. Then set v_0 ← v, i ← i_0 ← 0, a ← a_0 ← ARCS(v ), TAG(v ) ← 1, l ← 1, and go to R4.

R2. [Enter level l.] (At this point i = il−1, v = vi, and a = al−1 is an arc from v to vl−1.) If l = n, visit the solution v_0v_1 ...vn−1 and set l ← n − 1.

R3. [Advance a.] Set a ← NEXT(a ), the next neighbor of v.

R4. [Done with level?] If a ≠ Λ, go to R5. Otherwise if i = l − 1, go to R6. Otherwise set i ← i +1, v ← vi, a ← ARCS(v ).

R5. [Try a.] Set u ← TIP(a ) and tag u. If TAG(u ) > 1, return to R3. Otherwise set il ← i, al ← a, vl ← u, l ← l + 1, and go to R2.

R6. [Backtrack.] Set l ← l − 1, and stop if l = 0. Otherwise set i ← il, v ← vi. Untag all neighbors of vk, for l ≥ k > i. Then set a ← NEXT(al ); while a ≠ Λ, untag TIP(a ) and set a ← NEXT(a ). Finally set a ← al and return to R3.

This instructive algorithm differs subtly from the conventional structure of Algorithm B. Notice in particular that TIP(al ) is not untagged in step R6; that vertex won’t be untagged and chosen again until some previous decision has been reconsidered.

76. Let G have N vertices. For 1 ≤ k ≤ N, perform Algorithm R on the kth vertex v of G, except that step R1 should tag the first k − 1 vertices so that they are excluded. (You’ll need to make it work when n = 1. A tricky shortcut can be used: If we untag all neighbors of v = v_0 after Algorithm R stops, the net effect will be to tag only v.)

The n-omino placement counts 1, 4, 22, 113, 571, 2816, 13616, 64678, 302574 are computed almost instantly, for small n. (Larger n are discussed in Section 7.2.3.)

77. (a) All but the 13th and 18th, which require an upward or leftward step.

(b) True. If u ∈ H and u ≠ v, let pu be any node of H that’s one step closer to v.

(c) Again true: The oriented spanning trees are also ordinary spanning trees.

(d) The same algorithm works, except that step R4 must return to itself after setting a ← ARCS(v ). (We can no longer be sure that ARCS(v ) ≠ Λ.)

78. Extend Algorithm R to terminate immediately if WT(v ) ≥ U, otherwise to visit the singleton solution v. Also set w ← WT(v ) in step R1. Replace steps R2 and R5 by

R2′. [Enter level l]. If w ≥ L, visit the solution v_0v_1 ...vl−1.

R5′. [Try a.] Set u ← TIP(a ) and tag u. If TAG(u ) > 1 or w + WT(u ) ≥ U, return to R3. Otherwise set il ← i, al ← a, vl ← u, w ← w + WT(u ), l ← l + 1, and go to R2.

In step R6, set w ← w − WT(vl ) just before setting i ← il.

79. (a) (0,j) and (1,j) for j ≥ 44; (2,j) for j ≥ 32; (4,j), (8,j), (10,j) for j < 12.

(b) True, each of the Boolean functions ri,j is clearly monotone.

(c) The “couplers” can be simulated by playing sj∗ and gj∗ instead of sj and gj (as if the organist had assistants). Therefore the problem can be factored into independent subproblems for the Pedal, Swell, and Great separately: Let there be Pn, Sn, Gn playable sounds on the Pedal, Swell, and Great, and define P (z) = Σn Pnzn, S(z) = Σn Snzn, G(z) = Σn Gnzn; then Q(z) = Σn Qnzn is the convolution P (z)S(z)G(z).

(d) p_0 = p_12 = c_0 = c_1 = c_15 = 1 gives (0, 0), (0, 12), (0, 24), (1, 0), (1, 12); s_0 = s_19 = s_28 = c_3 = c_4 = 1 gives (the beautiful) (3, 0), (3, 19), (3, 28), (4, 19), (4, 28); etc.

(e) It’s unplayable if and only if i ∈ {2, 14, 15} or i′ ∈ {0, 1, 2, 14, 15} or (i ≠ i′ and either 3 ≤ i, i′ ≤ 8 or 9 ≤ i, i′ ≤ 15).

(f) Q_1 = 812 − 112 = 700, because we can’t have (14,j) or (15,j) without (13,j).

(g) Q_811 = 12 sounds lack only one pipe: With all inputs 1 except pj, for 12 ≤ j < 24, only r2,j is 0. (Thankfully there isn’t enough wind pressure to actually play this.)

(h) Brute-force backtrack programs can be written, using the monotonicity property (b) for cutoffs, in order to check small values and to list the actual sounds. But the best way to compute Pn, Sn, Gn, and Qn is to use generating functions.

For example, let G(z) = G_0(z) + G_1(z) + ... + G_63(z), where Gk(z) for k = (c_14c_13c_12c_11c_10c_9)2 enumerates the sounds for a given setting of console switches, excluding sounds already enumerated by Gj(z) for j < k. Then G_0(z) = 1; Gk(z) = 0 if c_13c_14 = 1; otherwise Gk(z) = f(c_9 + c_11 + c_12 + c_13 + 3c_14) when c_10 = 0, and Gk(z) = g(c_9 +1+ c_11 + c_12 + c_13 +3c_14, 1+ c_11 + c_12 + c_13 +3c_14) when c_10 = 1, where
f(n)=(1+zn)56−1,g(m,n)=(1+zn)12((1+zm)44−1).

Thus G(z) = 1 + 268z + 8146z^2+ 139452z^3+ ... + 178087336020z^10+ ... +12z^374+ z^380.

Similarly, with S(z)=Σk=063⁢ Sk(z) and k = (c_8c_7c_6c_5c_4c_3)2, we have S_0(z) = 1; S_32(z) = (1 + z)44 − 1; otherwise Sk(z) = f(c_3 + c_5 + c_6 + c_7) when c_4 = c_8 = 0, Sk(z) = g(c_3 + c_4 + c_5 + c_6 + c_7 + c_8, max(c_3,c_4) + c_5 + c_6 + c_7) when c_4 + c_8 > 0. Thus S(z) = 1 + 312z + 9312z^2+ 155720z^3+ ... + 180657383126z^10+ ... +12z^308+ z^312. [Curiously we have Sn > Gn for 1 ≤ n ≤ 107.]

The generating functions for P(z)=Σk=031⁢ Pk(z), with k = (c_16c_15c_2c_1c_0)2, are trickier. Let h(w, z) = (1+3wz^2 +2w^2z^3 +w^2z^4 +w^3z^4)8((1+2wz^2 +w^2z^3)4 −1). Then P_31(z) = h(z, z^2), and there are three main cases when 0 < k < 31: If c_0c_15 = c_1c_16 = 0, then Pk(z) = (1+ zc_15+c_16)32 − (1 + zc_15+c_16)20 if c_0 + c_1 + c_2 = 0, otherwise Pk(z) = (1 + zc_0+c_1+c_2+c_15+c_16)32 − 1. If c_0 = c_15, c_1 = c_16, c_2 = 0, then Pk(z) = q(zc_0+c_1),
q(z)=(1+3z2+2z3+z4)8(1+2z2+z3)4−2(1+2z2+z3)8(1+z2)4+(1+z2)8.

Otherwise we have Pk(z) = h(zc_0+c_1+c_2+c_15+c_16−2,z). Thus P (z) = 1+120z +2336z^2+ 22848z^3 + ... + 324113168z^10 + ... +8z^119 + z^120. And Q(z) = 1 + 700z + 173010z^2 + 18838948z^3+ 1054376915z^4+ 38386611728z^5+ 1039287557076z^6+ 22560539157160z^7+ 410723052356833z^8+ 6457608682396156z^9+ 89490036797524716z^10+ ... +12z^811+ z^812. So (Q2/(8122),…,Q10/(81210)≈(.5,.2,.06,.01,.003,.0005,.00009,.00002,.000003).

Dr Pell was wont to say, that in the Resolution of Questiones, the main matter is the well stating them: which requires a good mother-witt & Logick: as well as Algebra: for let the Question be but well-stated, and it will worke of it selfe: . . . By this way, an man cannot intangle his notions, & make a false Steppe.

— JOHN AUBREY, An Idea of Education of Young Gentlemen (c. 1684)

Section 7.2.2.1

1. (a) Note first that Algorithm 6.2.2T has its own LLINK and RLINK fields, for left and right children; they shouldn’t be confused with the links of the doubly linked list. After all deletions are done, LLINK(k) will be the largest search-tree ancestor of k that’s less than k; RLINK(k) will be the smallest ancestor of k that’s greater than k; but if there’s no such ancestor, the link will be 0. (For example, in Fig. 10 of Section 6.2.2, RLINK(LEO) would be PISCES and LLINK(AQUARIUS) would be the list head.)

(b) There are Cn=(2nn)1n+1 classes (the Catalan number), one for each binary tree.

(c) The size of each class is the number of topological sortings of the partial order generated by the relations k ≺ LLINK(k), k ≺ RLINK(k). And this number equals 1 only in the 2n−1 “degenerate” trees of height n (see exercise 6.2.2–5).

2. (a) (Solution by X. Lou.) We can prove that LLINK(ak) = ak − 1 and RLINK(ak) = (ak + 1) mod (n + 1) when ak is undeleted; hence that undeletion sets RLINK(ak − 1) and LLINK((ak + 1) mod (n + 1)) to the correct value ak. (If ak − 1 wasn’t deleted before ak, LLINK(ak) never changed. Otherwise LLINK(ak) became ak − 1 when ak − 1 was undeleted, by induction on k. A similar argument works for RLINK.) Notice that each LLINK and RLINK is reset exactly once, except that LLINK(1) and RLINK(n) remain 0.

(Programmers are advised to use this amazing fact only with great care, because the lists are malformed during the process and fully reconstructed only at the end.)

(b) No. For example, delete 1, 2, 3; then undelete 1, 3, 2.

(c) Yes. The argument of (a) applies to each maximal interval of affected elements.

3. (a) (x_1,...,x_6) = (1, 0, 0, 1, 1, 0). (In general the solutions to linear equations won’t always be 0 or 1. For example, the equations x_1+x_2 = x_2+x_3 = x_1+x_3 = 1 imply that x1=x2=x3=12; hence the corresponding exact cover problem is unsolvable.)

(b) In practice, m is much larger than n. Example (5) is just a “toy problem”! The best we can hope to achieve from n simultaneous equations is to express n of the variables in terms of the other m − n; that leaves 2m − n cases to try.

4. If G is bipartite, the exact covers are the ways to choose the vertices of one part. (Hence there are 2k solutions, if G has k components.) Otherwise there are no solutions. (Algorithm X will discover that fact quickly, although Algorithm 7B is faster.)

5. Given a hypergraph, find a set of vertices that hits each hyperedge exactly once. (In an ordinary graph this is the scenario of exercise 4.)

Similarly, the so-called hitting set problem is dual to the vertex cover problem.

6. The header nodes, numbered 1 through N, are followed by L ordinary nodes and M + 1 spacers; hence the final node Z is number L + M + N + 1. (There also are N + 1 records for the horizontal list of items; those “records” aren’t true “nodes.”)

7. Node 23 is a spacer; ‘−4’ indicates that it follows the 4th option. (Any nonpositive number would work, but this convention aids debugging.) Option 5 ends at node 25.

8. (Secondary items, which are introduced in the text after (24), are also handled by the steps below. Such items should occur after all of the primary items have been listed on the first line, and separated from them by some distinguishing mark.)

I1. [Read the first line.] Set N_1 ← −1, i ← 0. Then, for each item name α on the first line, set i ← i + 1, NAME(i) ← α, LLINK(i) ← i − 1, RLINK(i − 1) ← i. If α names the first secondary item, also set N_1 ← i − 1. (In practice α is limited to at most 8 characters, say. One should report an error if α = NAME(j) for some j < i.)

I2. [Finish the horizontal list.] Set N ← i. If N_1 < 0 (there were no secondary items), set N_1 ← N. Then set LLINK(N + 1) ← N, RLINK(N) ← N + 1, LLINK(N_1 + 1) ←N + 1, RLINK(N + 1) ← N_1 + 1, LLINK(0) ← N_1, RLINK(N_1) ← 0. (The active secondary items, if any, are accessible from record N + 1.)

I3. [Prepare for options.] Set LEN(i) ← 0 and ULINK(i) ← DLINK(i) ← i for 1 ≤ i ≤ N. (These are the header nodes for the N item lists, which are initially empty.) Then set M ← 0, p ← N + 1, TOP(p) ← 0. (Node p is the first spacer.)

I4. [Read an option.] Terminate with Z ← p if no input remains. Otherwise let the next line of input contain the item names α_1 ... αk, and do the following for 1 ≤ j ≤ k: Use an algorithm from Chapter 6 to find the index ij for which NAME(ij) = αj. (Report an error if unsuccessful. Complain also if an item name appears more than once in the same option, because a duplicate might make Algorithm X fail spectacularly.) Set LEN(ij) ← LEN(ij) + 1, q ← ULINK(ij), ULINK(p + j) ← q, DLINK(q) ← p + j, DLINK(p + j) ← ij, ULINK(ij) ← p + j, TOP(p + j) ← ij.

I5. [Finish an option.] Set M ← M +1, DLINK(p) ← p+k, p ← p+k+1, TOP(p) ← −M, ULINK(p) ← p − k, and return to step I4. (Node p is the next spacer.)

9. Set θ ← ∞, p ← RLINK(0). While p ≠ 0, do the following: Set λ ← LEN(p); if λ<θ set θ ← λ, i ← p; and set p ← RLINK(p). (We could exit the loop immediately if θ = 0.)

10. If LEN(p) > 1 and NAME(p) doesn’t begin with ‘#’, set λ ← M + LEN(p) instead of LEN(p). (Similarly, the “nonsharp preference” heuristic favors nonsharp items.)

11. Item a is selected at level 0, trying option x_0 = 12, ‘a d g’, and leading to (7). Then item b is selected at level 1, trying x_1 = 16, ‘b c f’. Hence, when the remaining item e is selected at level 2, it has no options in its list, and backtracking becomes necessary. Here are the current memory contents — substantially changed from Table 1:

i:

0

1

2

3

4

5

6

7

NAME(i):

—

a

b

c

d

e

f

g

LLINK(i):

0

0

0

0

3

0

5

6

RLINK(i):

0

2

3

5

5

0

0

0

x:

0

1

2

3

4

5

6

7

LEN(x):

—

2

1

1

1

0

0

1

ULINK(x):

—

20

16

9

27

5

6

25

DLINK(x):

—

12

16

9

27

5

6

25

x:

8

9

10

11

12

13

14

15

TOP(x):

0

3

5

—1

1

4

7

—2

ULINK(x):

—

3

5

9

1

4

7

12

DLINK(x):

10

3

5

14

20

21

25

18

x:

16

17

18

19

20

21

22

23

TOP(x):

2

3

6

—3

1

4

6

—4

ULINK(x):

2

9

6

16

12

4

18

20

DLINK(x):

2

3

6

22

1

27

6

25

x:

24

25

26

27

28

29

30

TOP(x):

2

7

—5

4

5

7

—6

ULINK(x):

16

7

24

4

10

25

27

DLINK(x):

2

7

29

4

5

7

—

12. Report that x is out of range if x ≤ N or x > Z or TOP(x) ≤ 0. Otherwise set q ← x and do “print ‘NAME(TOP(q))’ and set q ← q + 1; if TOP(q) ≤ 0 set q ← ULINK(q)” until q = x. Then set i ← TOP(x), q ← DLINK(i), and k ← 1. While q ≠ x and q ≠ i, set q ← DLINK(q) and k ← k + 1. If q ≠ i, report that the option containing x is ‘k of LEN(i)’ in item i’s list; otherwise report that it’s not in that list.

[Algorithm C extends Algorithm X to colors. If COLOR(q) ≠ 0, also print ‘:c’ where c = COLOR(q) if COLOR(q) > 0, otherwise c = COLOR(TOP(q)).]

13. For 0 ≤ j < l, node xj is part of an option in the solution. By setting r ← xj and then r ← r + 1 until TOP(r) < 0, we’ll know exactly what that option is: It’s option number −TOP(r), which begins at node ULINK(r). (Many applications of Algorithm X have a custom-made output routine, to convert x_0 ... xl−1 into an appropriate format — presenting it directly as a sudoku solution or a box packing, etc.)

Exercise 12 explains how to provide further information, not only identifying the option of xj but also showing its position in the search tree.

14. (a) The options are ‘Sj Mk’, for all 0 ≤ j, k < n except j = k or j = (k + 1) mod n.

(b) There are (u_3,...,u_10) = (1, 2, 13, 80, 579, 4738, 43387, 439792) solutions. The running time for n = 10 is about 180 (or 275) mems per solution with (or without) MRV.

[This problem has a rich history: E. Lucas presented and named it in his Théorie des Nombres (1891), 215, 491–495. An equivalent problem had, however, already been posed by P. G. Tait, and solved by A. Cayley and T. Muir; see Trans. Royal Soc. Edinburgh 28 (1877), 159, and Proc. Royal Soc. Edinburgh 9 (1878), 338–342, 382– 391, 11 (1880), 187–190. In particular, Muir found the recurrence relation

(n−1)un+1=(n2−1)un+(n+1)un−1+(−1)n⋅4,for n>1.

Clearly u_2 = 0; a careful consideration of initial values shows that the choices u_0 = 1 and u_1 = −1 give mathematically clean expressions, such as the explicit formula
un=Σk=0n(−1)k2n2n−k(2n−kk)(n−k)!.

(See J. Touchard, Comptes Rendus Acad. Sci. 198 (Paris, 1934), 631–633; I. Kaplansky, Bull. Amer. Math. Soc. 49 (1943), 784–785.) The kth term of this formula can also be written n! Σj(−1)j+k2k−2j/((k−2j)!j!(n−1)j¯); hence we have the curious identity
unn!=Σj=0n/2(−1)jj!Tn−2j(n−1)j¯=Tn−Tn−1n−1+Tn−2/2!(n−1)(n−2)−Tn−3/3!(n−1)(n−2)(n−3)+⋅⋅⋅,
where Tn=Σk=0n(−2)k/k! is the sum of the first n + 1 terms of the power series for e−2. The ménage numbers therefore satisfy the interesting asymptotic formula
un=n!e2=(1−1n−1+1/2!(n−1)(n−2)+⋅⋅⋅+(−1)k/k!(n−1)⋅⋅⋅(n−k)+O(n−k−1))

for all fixed k ≥ 0, discovered by I. Kaplansky and J. Riordan (Scripta Mathematica 12 (1946), 113–124). In fact, M. Wyman and L. Moser proved that the sum of this series for 0 ≤ k < n differs from un by less than 1/2 (Canadian J. Math. 10 (1958), 468–480). Among many other things, they also found a (complicated) expression for the exponential generating function ∑n unzn/n!. The ordinary generating function ∑n unzn has the surprisingly nice form ((1 − z)/(1 + z))F(z/(1 + z)2), where F(z) = ∑n≥0 n!zn; see P. Flajolet and R. Sedgewick, Analytic Combinatorics (2009), 368–372.]

15. Omit the options with i = n − [n even] and j > n/2.

(Other solutions are possible. For example, we could omit the options with i = 1 and j ≥ n; that would omit n − 1 options instead of only ⌊n/2⌋. However, the suggested rule turns out to make Algorithm X run about 10% faster.)

16. The two solutions are ‘r_1 c_2 a_3 b−1’ ‘r_2 c_4 a_6 b−2’ ‘r_3 c_1 a_4 b_2’ ‘r_4 c_3 a_7 b_1’ ‘a_2’ ‘a_5’ ‘a_8’ ‘b−3’ ‘b_0’ ‘b_3’; ‘r_1 c_3 a_4 b−2’ ‘r_2 c_1 a_3 b_1’ ‘r_3 c_4 a_7 b−1’ ‘r_4 c_2 a_6 b_2’ ‘a_2’ ‘a_5’ ‘a_8’ ‘b−3’ ‘b_0’ ‘b_3’. At the top levels, the MRV heuristic causes Algorithm X to branch first on the slack variables a_2, a_8, b−3, and b_3, which each have at most two possibilities. (And that’s actually a pretty strange way to tackle the four queens problem!)

17. Branch first on r_3, which has four options. If ‘r_3 c_1 a_4 b_2’, there’s just one option for c_2, then c_3, then r_2, so we get the first solution: ‘r_3 c_1 a_4 b_2’ ‘r_1 c_2 a_3 b−1’ ‘r_4 c_3 a_7 b_1’ ‘r_2 c_4 a_6 b−2’. If ‘r_3 c_2 a_5 b_1’, c_3 is forced, then r_2 can’t be covered. If ‘r_3 c_3 a_6 b_0’, r_2 is forced, then c_2 can’t be covered. If ‘r_3 c_4 a_7 b−1’, we cruise to the second solution: ‘r_3 c_4 a_7 b−1’ ‘r_1 c_3 a_4 b−2’ ‘r_2 c_1 a_3 b_1’ ‘r_4 c_2 a_6 b_2’. (And that’s a good way.)

18. ‘c e’ ‘a d f’ ‘b g’ (as before) and ‘b c f’ ‘a d g’ (new).

19. When all primary items have been covered in step X2, accept a solution only if LEN(i) = 0 for all of the active secondary items, namely the items accessible from RLINK(N + 1). [This algorithm is called the “second death” method, because it checks that all of the purely secondary options have been killed off by primary covering.]

20. For 1 ≤ k < m, set t ← k & (−k); include secondary item yk in option αj for k ≤ j < min(m, k + t) and in option βj for k − t ≤ j < k.

Equivalently, to set up option αj, include a and set t ← j; while t > 0, include yt and set t ← t & (t − 1). To set up option βj, include b and set t ← −1 − j; while t > −m, include y−t and set t ← t & (t − 1).

If j > k, options αj and βk both contain yj&−2⌊lg(j − k)⌋.

21. The options αji will contain the primary item ai. Simply do k − 1 pairwise orderings, with secondary items yki to ensure that jk ≤ jk+1. If m is a power of 2, it turns out that the options for 1 < i < k each have exactly lg m secondary items. For example, if m = 4 and k > 2, the options αj2 are ‘a_2 y11 y21’, ‘a_2 y21 y12’, ‘a_2 y31 y22’, ‘a_2 y32 y22’.

(The author attempted to knock out options for αi′ with i′ < i − 1 or i′ > i + 1, by adding additional secondary items, but that turned out to be a bad idea.)

Of course, this method doesn’t compete with the lightning-quick methods for combination generation in Section 7.2.1.3. For instance, when m = 20 and k = 8 it needs 1.1 Gμ to crank out the (278)=2220075 coverings, about 500 mems per solution.

22. (a) Let n′ = ⌊n/2⌋ + 1. By rotation/reflection we can assume that the queen in column n′ (the middle column) is in row i and the queen in row n′ is in column j, where 1 ≤ i < j < n′. We obtain a suitable exact cover problem by leaving out the options o(i, j) = ‘ri cj ai+j bi−j’ for i = j or i + j = n + 1; also omit o(i, j) for i > j when j = n′; j > i when i = n′; and (i, j) = (n′ − 1, n′) or (n′, 1). Then include secondary items to force the pairwise ordering of αk = o(k + 1, n′) and βk = o(n′, k + 2), for 0 ≤ k < m = n′ − 2.

(b) Now we assume a queen in (j, j), where 1 ≤ j < n′, and that the queen in row n is closer to the bottom right corner than the queen in column n. So we omit options o(i, j) for i + j = n + 1 or i = j ≥ n′ or (i, j) = (n, 2) or (i, j) = (n −1, n); we make item b_0 primary; and we let αk = o(n, n − k − 1), βk = o(n − k − 2, n) for 0 ≤ k < m = n − 3.

(c) This time we want queens in (i, i) and (j, n + 1 − j) where 1 ≤ i < j < n′. We promote an+1 and b_0 to primary; omit o(i, j) when i = j ≥ n′ − 1 or i = n + 1 − j ≥ n′ or (i, j) = (1, n); and let αk = o(k + 1, k + 1), βk = o(k + 2, n − k − 1) for 0 ≤ k < m = n′ − 2.

In case (a) there are (0, 0, 1, 8, 260, 9709, 371590) solutions for n = (5, 7,..., 17); Algorithm X handles n = 17 in 3.4 Gμ. [In case (b) there are (0, 0, 1, 4, 14, 21, 109, 500, 2453, 14498, 89639, 568849) for n = (5, 6,..., 16); and n = 16 costs 6.0 Gμ. In case (c), similarly, there are (1, 0, 3, 6, 24, 68, 191, 1180, 5944, 29761, 171778, 1220908) solutions; n = 16 costs 5.5 Gμ.]

23. (a) Consider the queens in column a of row 1, row b of column n, column c¯ of row n, and row d¯ of column 1, where x¯=n+1−x. (These four queens are distinct, because no queen is in a corner. Notice also that neither a¯ nor b¯ nor c¯ nor d¯ can equal a.) Repeated rotations and/or reflections will change these numbers from (a, b, c, d) to

(b,c,d,a), (c,d,a,b), (d,a,b,c), (d¯,c¯,b¯,a¯), (c¯,b¯,a¯,d¯), (b¯,a¯,d¯,c¯), (a¯,d¯,c¯,b¯).

Those eight 4-tuples are usually distinct, and in such cases we can save a factor of 8 by eliminating all but one of them. There always is a solution with a≤b,c,d>a¯; and those inequalities can be enforced by doing three simultaneous pairwise comparisons, between the options for row 1 and the respective options for column n, row n, and column 1. For example, the options that correspond to a = 1 when n = 16 are ‘r_1 c_2 a_3 b−1’; ‘r_2 c_16 a_18 b−14 x_1 x_2 x_4’; ‘r_15 c_16 a_31 b−1 x_1 x_2 x_4’; ‘r_16 c_2 a_18 b_14 y_1 y_2 y_4’; ‘r_16 c_14 a_30 b_2 y_1 y_2 y_4’; ‘r_2 c_1 a_3 b_1 z_1 z_2 z_4’; ‘r_15 c_1 a_16 b_14 z_1 z_2 z_4’. (Here m = n/2 − 1 = 7.)

With this change, the number of solutions for n = 16 drops from 454376 to 64374 (ratio ≈ 7.06), and the running time drops from 4.3 Gμ to 1.2 Gμ (ratio ≈ 3.68).

[The author experimented with further restrictions, so that solutions were allowed only if (i) a < b, c, d; (ii) a = b < c, d; (iii) a = b = c < d; (iv) a = b = c = d; (v) a = c < b, d. Five options were given for each value of a < n/2 − 1, and m was 6 instead of 7. The number of solutions decreased to 59648; but the running time increased to 1.9 Gμ. Thus a point of diminishing returns had been reached. (A completely canonical reduction would have produced 57188 solutions, with considerable difficulty.)]

(b) This case is almost identical to (a), because the queen in the center vacates all other diagonal cells. Requiring a≤b,c,d>a¯ reduces the number of solutions for n = 17 from 4067152 to 577732 (ratio ≈ 7.04), and run time to 3.2 Gμ (ratio ≈ 4.50).

24. We simply combine compatible options into (a) pairs, (b) quadruplets, and force a queen in the center when n is odd. For example, when n = 4 we replace (23) by (a) ‘r_1 c_2 a_3 b−1 r_4 c_3 a_7 b_1’; ‘r_1 c_3 a_4 b−2 r_4 c_2 a_6 b_2’; ‘r_2 c_1 a_3 b_1 r_3 c_4 a_7 b−1’; ‘r_2 c_4 a_6 b−2 r_3 c_1 a_4 b_2’; (b) ‘r_1 c_2 a_3 b−1 r_2 c_4 a_6 b−2 r_4 c_3 a_7 b_1 r_3 c_1 a_4 b_2’; ‘r_2 c_1 a_3 b_1 r_3 c_4 a_7 b−1 r_1 c_3 a_4 b−2 r_4 c_2 a_6 b_2’. The options when n = 5 are (a) ‘r_1 c_2 a_3 b−1 r_5 c_4 a_9 b_1’; ‘r_1 c_4 a_5 b−3 r_5 c_2 a_7 b_3’; ‘r_2 c_1 a_3 b_1 r_4 c_5 a_9 b−1’; ‘r_2 c_5 a_7 b−3 r_4 c_1 a_5 b_3’; ‘r_3 c_3 a_6 b_0’; (b) ‘r_1 c_2 a_3 b−1 r_2 c_5 a_7 b−3 r_5 c_4 a_9 b_1 r_4 c_1 a_5 b_3’; ‘r_2 c_1 a_3 b_1 r_1 c_4 a_5 b−3 r_4 c_5 a_9 b−1 r_5 c_2 a_7 b_3’; ‘r_3 c_3 a_6 b_0’.

An n-queen solution is either asymmetric (changed by 180° rotation) or singly symmetric (changed by 90° rotation but not 180°) or doubly symmetric (unchanged by 90° rotation). Let Qa(n), Qs(n), Qd(n) be the number of such solutions that are essentially different; then Q(n) = 8Qa(n) + 4Qs(n) + 2Qd(n) when n > 1. Furthermore there are 4Qs(n) + 2Qd(n) solutions to (a) and 2Qd(n) solutions to (b). Hence we can determine the individual values just by counting solutions, and we obtain these results for small n:

n=4567891011121314151617Qa(n)=01041142893291765919745647284743184618911975869Qs(n)=00121431218321053107342006Qd(n)=1100000044003264

We can reduce the solutions to (a) by a factor of 2, by simply eliminating the options that contain {r_1, ck} for k ≥ ⌈n/2⌉. We can reduce the solutions to (b) by a factor of 2⌊n/4⌋, by simply eliminating the options that contain {rj, ck} for j < ⌈n/2⌉ and k ≥ ⌈n/2⌉. With these simplifications, the computation of Qd(16) needs only 70 Kμ; and then the computation of Qs(16) needs only 5 Mμ. Only 20 Mμ are needed to determine that Qd(32) = 2^7 · 1589.

25. With 64 items, one for each cell of the chessboard, let there be 92 options, one for each of the 92 solutions to the eight queens problem (see Fig. 68). Every option names eight of the 64 items; so an 8-coloring is equivalent to solving this exact cover problem. Algorithm X needs only 25 kilomems and a 7-node search tree to show that such a mission is impossible. [In fact no seven solutions can be disjoint, because each solution touches at least three of the twenty cells 13, 14, 15, 16, 22, 27, 31, 38, 41, 48, 51, 58, 61, 68, 72, 77, 83, 84, 85, 86. See Thorold Gosset, Messenger of Mathematics 44 (1914), 48. However, Henry E. Dudeney found the illustrated way to occupy all but two cells, in Tit-Bits 32 (11 September 1897), 439; 33 (2 October 1897), 3.]

12345678

78563412

46718235

23854167

84236751

51672384

67481523

512784

26. This is an exact cover problem with 92 + 312 + 396 + ··· + 312 = 3284 options (see exercise 7.2.2–6). Algorithm X needs about 32 megamems to find the solution shown, and about 1.3 Tμ to find all 11,092 of them.

07348652

18650437

75421860

26835071

34072186

52183704

80564213

61207345

27. Let ujh and djh be secondary items for 1 ≤ j ≤ 2n and 1 ≤ h ≤ ⌊n/2⌋. Insert the gadget
uj1 uj2 ⋅⋅⋅ uj ⌈i/2⌉ u(j+1) ⌈i/2⌉ ⋅⋅⋅ uk ⌈i/2⌉ ⋅⋅⋅ uk2 uk1
into each option (16); also append similar options, but with ‘u’ changed to ‘d’, except when i = n. [Solutions whose planar graph “splits” will be obtained more than once. One such example is 12 10 8 6 4 11 9 7 5 4 6 8 10 12 5 7 9 11 3 1 2 1 3 2.]

28. (a) Denoting that formula by ρ(c_0, t_0; ... ; cl, tl), notice that if cj′=tj+1−cj we have ρ(c0,t0;…;cl,tl)+ρ(c0′,t0;…;cl′,tl)=1. Consequently the completion ratio is 1/2 if and only if cj′=cj for all j, namely when tj = 2cj − 1.

(b) The ratio ρ(c_0, t_0; ... ; cl, tl) never has an odd denominator, because p/q + p′/q′ has an even denominator whenever q and p′ are odd and q′ is even. But we can get arbitrarily close to 1/3, since ρ(2, 4; ... ;2, 4) = 1/3 + 1/(24 · 4l).

29. If T has only a root node, let there be one column, no rows. Otherwise let T have d ≥ 1 subtrees T_1, ... , Td, and assume that we’ve constructed matrices with rows Rj and columns Cj for each Tj. Let C = C_1 ⋃ ··· ⋃Cd. The matrix for T is obtained by appending three new columns {0, 1, 2} and the following new rows: (i) ‘0 1 2 and all columns of C\Cj’, for 1 ≤ j ≤ d; (ii) ‘j and all columns of C’, for j ∈ {0, 1}. The matrix for the example tree has 15 columns and 14 rows.

011111000000000

101111000000000

110111000000000

111100000000000

111010000000000

000000011111000

000000101111000

000000110111000

000000111100000

000000111010000

000000111111111

111111000000111

111111111111100

111111111111010

30. Yes, assuming that duplicate options are permitted. Use the previous construction, but change ‘C\Cj’ t o‘C’ if Tj is a solution node. (Without duplicate options, no two solution nodes can be siblings.)

31. (a) In step I4 of answer 8, insert p+j into the rth position of the list for ij, instead of at the bottom, where r is uniform between 1 and LEN(ij).

(b) In answer 9, when λ < θ also set r ← 1; when λ = θ, set r ← r + 1, and change i ← p with probability 1/r.

32. (a) No. Otherwise there would be an option with no primary items.

(b) Yes, but only if there are two options with the same primary items.

(c) Yes, but only if there are two options whose union is also an option, when restricted to primary items.

(d) The number of places, j, where x = 1 and x′ = 0 must be the same as the number where x = 0 and x′ = 1. For if A has exactly k primary items in every option, exactly jk primary items are being covered in different ways.

(e) Again distances must be even, because every solution also solves the restricted problem, which is uniform. (Consequently it makes sense to speak of the semidistance d(x, x′)/2 between solutions of a quasi-uniform exact covering problem. The semidistance in a polyform packing problem is the number of pieces that are packed differently.)

33. (Solution by T. Matsui.) Add one new column at the left of A, all 0s. Then add two rows of length n + 1 at the bottom: 10 ... 0 and 11 ... 1. This (m + 2) × (n + 1) matrix A′ has one solution that chooses only the last row. All other solutions choose the second-to-last row, together with rows that solve A.

34. (Solution by T. Matsui.) Assume that all 1s in column 1 appear in the first t rows, where t > 3. Add two new columns at the left, and two new rows 1100 ... 0, 1010 ... 0 of length n + 2 at the bottom. For 1 ≤ k ≤ t, if row k was 1αk, replace it by 010αk if k ≤ t/2, 011αk if k > t/2. Insert 00 at the left of the remaining rows t + 1 through m.

This construction can be repeated (with suitable row and column permutations) until no column sum exceeds 3. If the original column sums were (c_1,...,cn), the new A′ has 2T more rows and 2T more columns than A did, where T=Σj=1n(cj∸3).

One consequence is that the exact cover problem is NP-complete even when restricted to cases where all row and column sums are at most 3.

Notice, however, that this construction is not useful in practice, because it disguises the structure of A: It essentially destroys the minimum remaining values heuristic, because all columns whose sum is 2 look equally good to the solver!

35. Take a matrix with column sums (c_1,...,cn), all ≤ 3, and extend it with three columns of 0s at the right. Then add the following four rows: (x_1,...,xn, 0, 1, 1), (y_1,...,yn, 1, 0, 1), (z_1,...,zn, 1, 1, 0), and (0,...,0, 1, 1, 1), where xj = [cj < 3], yj = [cj < 2], zj = [cj < 1]. The bottom row must be chosen in any solution.

36. The following modifications (which work also with Algorithm C) will find all solutions in lexicographic order; we can terminate early if we want only the first one.

Set LL ← 0 in step X1. (We will use the MRV heuristic, but only on levels > LL.)

If RLINK(0) = 0 and l = LL + 1 in step X2, visit the current solution as usual. Otherwise, however, set LL ← LL + 1 and do the following while l > LL (because the current solution was not found lexicographically): Set l ← l − 1, i ← TOP(xl); uncover the items ≠ i in the option that contains xl (as in X6); uncover i (as in X7).

In step X3, if l = LL simply set i ← RLINK(0). Otherwise use exercise 9, say.

If l < LL after setting l ← l − 1 in step X8, set LL ← l.

To get the lexicographically smallest solution to the n queens problem, make sure that the first n items are r_1, r_2, ... , rn. (The other primary items, cj, can follow in any order.) The first solution for n = 32, found after 4.2 Gμ, has queens in columns 1, 3, 5, 2, 4, 9, 11, 13, 15, 6, 18, 24, 26, 30, 25, 31, 28, 32, 27, 29, 16, 19, 10, 8, 17, 12, 21, 7, 14, 23, 20, 22. (Without MRV the computation would have taken 35.6 Gμ.)

[The analogous problem for n = 48 is already quite difficult; that case was first solved by Wolfram Schubert. The best results currently known for large n have been obtained via sophisticated methods of integer programming: In November 2017, Matteo Fischetti and Domenico Salvagnin were the first to solve the case n = 56 and many larger cases, although n = 62 was still unsolved; see arXiv:1907.08246 [cs.DS] (2019), 14 pages. See also OEIS A141843 for the latest developments.]

37. (a) Let ai,j = 0 if i ≤ 0 or j ≤ 0; otherwise
ai,j=mex({ai,j−k|k>0}∪{ai−k,j|k>0}∪{ai−k,j−k|k>0}∪{ai+k,j−k|k>0})

where ‘mex’ is defined in exercise 7.1.3–8. It is not difficult to verify that ai,qi = 1 and that each of the sequences 〈ai,n〉, 〈an,j〉 for n ≥ 1 is a permutation of the positive integers. (See OEIS A065188 and Alec Jones’s A269526.)

(b) The following exercise gives strong empirical evidence for this conjecture. And in the full plane, the analogous spiral sequence can be analyzed: See F. M. Dekking, J. Shallit, and N. J. A. Sloane, Electronic J. Combinatorics 27 (2020), #P1.52, 1–27.

38. The following method, inspired by Eq. 7.2.2–(6) and the previous exercise, uses binary vectors a, b, c, where c has both positive and negative subscripts.

G1. [Initialize.] Set r ← 0, s ← 1, t ← 0, n ← 0. (We’ve computed qk for 1 ≤ k ≤ n.)

G2. [Try for qn ≤ n.] (At this point ak = 1 for 1 ≤ k < s and as = 0; also ck = 1 for −r < k ≤ t and c−r = ct+1 = 0; each vector contains n 1s.) Set n ← n + 1, k ← s.

G3. [Found?] If k > n − r go to G4. Otherwise if ak = bk+n = ck−n = 0, go to G5. Otherwise set k ← k + 1 and repeat this step.

G4. [Make qn > n.] Set t ← t + 1, qn ← n + t, an+t ← b2n+t ← ct ← 1, and return to G2.

G5. [Make qn ≤ n.] Set qn ← k, ak ← bk+n ← ck−n ← 1. If k = s, set s ← s + 1 repeatedly until as = 0. If k = n − r, set r ← r + 1 repeatedly until c−r = 0. Return to G2.

In step G2 we have s ≈ n − r ≈ t ≈ n/ϕ; hence the running time is extremely short. Empirically, in fact, the calculation of qn requires at most 19 accesses to the bit vectors (averaging about 5.726 accesses), for each n. Agreement with exercise 37 is very close:

(q999999997,...,q1000000004)=(618033989,1618033985,618033988,)                                              1618033988,1618033990,1618033992,1618033994,618033991).

Moreover, it’s likely that qn ∈ [n/ϕ − 3 . . n/ϕ + 5] ⋃ [nϕ − 2 .. nϕ + 1] for all n.

39. (a) With probability (1 − p)n, no items will be selected; in such cases we must restart the clause generator, because options can’t be empty. Ten random trials with m = 500, n = 100, and p = .05 gave respectively (444, 51, 138, 29, 0, 227, 26, 108, 2, 84) solutions, costing about 100 megamems per solution.

Although the exercise did not call for a mathematical analysis, we can derive a formula for the expected number of solutions by computing the probability that a given subset of the options is an exact cover, then summing over all subsets. If the subset has k items, and if each item in each option were present with probability p, this probability would be (kp(1 − p)k−1)n. However, we’ve excluded empty options; the true probability f(n, p, k) turns out to be k!{nk}(p(1−p)k−1)n/(1−(1−p)n)k. The sum Σk{mk}f(n,p,k), when (m, n, p) = (500, 100, .05), is approximately 3736.96 with the incorrect formula and 297.041 with the correct one.

[In unpublished notes, Robin Pemantle and Boris Pittel have independently derived asymptotic results for m = αn and p = r/n, for fixed α and r as n → ∞. The behavior of Algorithm X with this random model is not easy to analyze, but an analysis may be within reach because of the recursive structure.]

(b) This case has completely different behavior. In the first place, n must obviously be a multiple of r. In the second place, we’ll need more options to get even one solution when n = 100 and r = 5, because conveniently small options don’t exist.

Proof: The total number of set partitions into twenty subsets of size 5 is P = 100!/(20! · 5!20) ≈ 10^98; the total number of possible options is N=(1005)=75287520. The probability that any particular set partition occurs as a solution is the probability that twenty given options occur in a random sample of m, with replacement, namely g(N,m,20)=Σk (20k)(−1)k(N−k)m/Nm=Σt {mt}t!(N−20t−20)/Nm. If m isn’t extremely large, this is almost the same as the probability without replacement, namely (N−20m−20)/(Nm)≈(m/N)20. The expected number of solutions when m = (500, 1000, 1500), respectively, is P g(N, m, 20) ≈ (.000002, 2.41, 8500).

40. Set fm ← 0 and fk−1 ← fk | rk for m ≥ k > 1. The bits of uk represent items that are being changed for the last time.

Let uk = u′ + u″, where u′ = uk & p. If uk ≠ 0 at the beginning of step N4, we compress the database as follows: For N ≥ j ≥ 1, if sj & u′ ≠ u′, delete (sj, cj); otherwise if sj & u″ ≠ 0, delete (sj, cj) and insert ((sj&u¯k) | u′,cj).

To delete (sj, cj), set (sj, cj) ← (sN , cN) and N ← N − 1.

When this improved algorithm terminates in step N2, we always have N ≤ 1. Furthermore, if we let pk = r_1 | ··· | rk−1, the size of N never exceeds 2νk, where νk = ν〈pk rk fk〉 is the size of the “frontier” (see exercise 7.1.4–55).

[In the special case of n queens, represented as an exact cover problem as in (23), this algorithm is due to I. Rivin, R. Zabih, and J. Lamping, Inf. Proc. Letters 41 (1992), 253–256. They proved that the frontier for n queens never has more than 3n items.]

41. The author has had reasonably good results using a triply linked binary search tree for the database, with randomized search keys. (Beware: The swapping algorithm used for deletion was difficult to get right.) This implementation was, however, limited to exact cover problems whose matrix has at most 64 columns; hence it could do n queens via (23) only when n < 12. When n = 11 its database reached a maximum size of 75,009, and its running time was about 25 megamems. But Algorithm X was noticeably better: It needed only about 12.5 Mμ to find all Q(11) = 2680 solutions.

In theory, this method will need only about 23n steps as n → ∞, times a small polynomial function of n. A backtracking algorithm such as Algorithm X, which enumerates each solution explicitly, will probably run asymptotically slower (see exercise 7.2.2–15). But in practice, a breadth-first approach needs too much space.

On the other hand, this method did beat Algorithm X on the n queen bees problem of exercise 7.2.2–16: When n = 11 its database grew to 364,864 entries; it computed H(11) = 596,483 in just 30 Mμ, while Algorithm X needed 440 Mμ.

42. The set of solutions for sj can be represented as a regular expression αj instead of by its size, cj. Instead of inserting (sj + t, cj) in step N3, insert αj k. If inserting (s, α), when (si, αi) is already present with si = s, change αi ← αi ∪ α. [Alternatively, if only one solution is desired, we could attach a single solution to each sj in the database.]

43. Let i = (i_1i_0)3 and j = (j_1j_0)3; then cell (i, j) belongs to box (i_1j_1)3. Mathematically, it’s cleaner to consider the matrices aij′=aij−1, bij′=bij−1, cij′=cij−1, which are the “multiplication tables” of interesting binary operators on {0,...,8}. We have aij′=((i0i1)3+j) mod 9; bij′=((i0+j1) mod 3, (i_1 + j_0) mod 3)3; and cij′=((i0+i1+j1) mod 3, (i_0 − i_1 + j_0) mod 3)3. (Furthermore the latter two operators are “isotopic”: cij′=b(iπ)(jπ−)′π, when (i_1, i_0)3π = (i_1, (i_0 + i_1) mod 3)3.)

[A pattern like (28c) appeared in a Paris newspaper of 1895, in connection with magic squares. But no properties of its 3×3 subsquares were mentioned; it was a sudoku solution purely by coincidence. See C. Boyer, Math. Intelligencer 29, 2 (2007), 63.]

44. No. The 33rd digit is 0. [A sudoku whose clues are π’s first 32 digits was first constructed by Johan de Ruiter in 2007; see www.puzzlepicnic.com/puzzle?346. Furthermore, π’s first 22 digits can actually be arranged in a circle to give a uniquely solvable sudoku, if we also require the elements of both main diagonals to be distinct! See Aad Thoen and Aad van de Wetering, Exotische Sudoku’s (2016), 144.]

45. Step X3 chooses p_44, p_84, p_74, p_24, p_54, p_14, p_82, p_42, p_31, p_32, p_40, p_45, p_46, p_50, p_72, p_60, p_00, p_62, p_61, p_65, p_35, p_67, p_70, p_71, p_75, p_83, p_13, p_03, p_18, p_16, p_07, p_01, p_05, p_15, p_21, p_25, p_76, p_36, p_33, p_37, p_27, p_28, p_53, p_56, p_06, p_08, p_58, p_77, p_88, in that order.

46. The lists for items p_44, p_84, r_33, r_44, r_48, r_52, r_59, r_86, r_88, c_22, c_43, b_07, b_32, b_39, b_43, b_54, and b_58 have length 1 when Algorithm X begins to tackle puzzle (29a). Step X3 will branch on whichever item was placed first in step X1. (The author’s sudoku setup program puts p before r before c before b in that step.)

47. r_13, c_03, b_03, b_24, b_49, b_69. The latter three were hidden already in (32).

48. In case (a) we list the available columns; in case (b) we list the available rows:

(Notice that “hidden” singles and pairs, etc., become “naked” in this representation. Similar plots, which relate boxes to values, are also possible; but they’re trickier, because boxes aren’t orthogonal to rows or columns.)

49. (a) For columns, remove all items rik and bxk, as well as cjk with j ≠ j_0; let uj — vk when an option contains ‘pij_0 cj_0k’. For boxes, remove all rik, cjk, and bxk with x ≠ x_0; let uj — vk when an option contains ‘p(3 ⌊ x_0/3 ⌋ + ⌊j/3⌋)(3(x mod 3)+(j mod 3)) bx_0k’.

(b) The n − q non-neighbors of a hidden q-tuple (e.g., {u_3, u_8, u_1}) are “naked.”

(c) By (b) it suffices to list the naked ones (and only those for which q < r). Let’s denote the option in (30) by ijk. In row 4 we find the naked pair {u_3, u_8}, hence we can delete options 411, 417, 421, 427, 471; also the naked triple {u_1, u_3, u_8}, so we can also delete option 424. There’s no nakedness in the columns. The naked triple {u_0, u_3, u_6} in box 4 allows deletion of options 341, 346, 347, 351, 356, 357.

(d) Let ui — vj if there’s an option that contains ‘rik_0 cjk_0’. When k_0 = 9 there’s a naked pair {u_1, u_5}, so we can delete options 079 and 279.

[Many other reductions have been proposed. For example, (33) has a “pointing pair” in box 4: Since ‘4’ and ‘8’ must occupy that box in row 3, we can remove options 314, 324, 328, 364, 368, 378. Classic references are the early tutorials by W. Gould, The Times Su Doku Book 1 (2005); M. Mepham, Solving Sudoku (2005). A comprehensive theory, applicable also to many other problems, has been developed by D. Berthier, Pattern-Based Constraint Satisfaction and Logic Puzzles (2012).]

50. Such a puzzle must add a 7 or 8 in one of 18 places, because (29c) has just 2 solutions. So there are 36 of them (18 isomorphic pairs).

51. We can solve this problem with Algorithm M, using options (30) with k ≠ 8 and giving multiplicity 2 to each of the items ri7, cj7, bx7. There are six solutions, all of which extend the partial solution shown. Only one yields a sudoku square when we change half of the 7s to 8s.

52. (Solution by F. Stappers.) Puzzles claiming to be “the world’s hardest sudoku” keep appearing in online forums. Rated by search tree size with Algorithm X, the toughest among nearly 27,000 such extreme puzzles is shown here in a canonical form. (It’s number 6539 in a list available from sites.google.com/site/sudoeleven/ (2011).) Its randomized search tree sizes are 24400±1900 — astonishingly high for sudoku; and its mean running time is about 12 Mμ.)

53. (a) Every shidoku solution is equivalent to one of the two special solutions A or B below (which incidentally have respectively 32 and 16 automorphisms, in the sense of exercise 114). We can’t uniquely specify either solution unless we have at least one clue in each of the regions {A, B, C, D} of C.

(b) Only 4^4 = 256 sets of four clues meet the conditions of (a), for each of A and B; we can test them all. Reducing by the automorphisms leaves two for A and eleven for B:

(There also are 22 essentially different shidoku puzzles with five irredundant clues, and a unique puzzle with six. The latter, which is solved by A, is shown above at the bottom left; it cannot omit a clue without having an empty region in either C or CT. These results were discovered by Ed Russell in 2006.)

54. For example, removing clues one at a time shows that only 10 of the 32 givens are actually essential. The best strategy for finding all minimal X is probably to examine candidate sets in order of decreasing cardinality: Suppose W ⊆ X, and suppose that previous tests have shown that the solution is unique, given X, but not given X \ w for any w ∈ W. Thus X is minimal if W = X. Otherwise let X\W = {x_1,...,xt}, and test X \ xi for each i. Suppose the solution turns out to be unique if and only if i > p. Then we schedule the t − p candidate pairs (W ⋃ {x_1,...,xp}, X \ xi), p < i ≤ t, for processing in the next round. With suitable caching of previous results, we can avoid testing the same subset of clues more than once. Furthermore we can readily modify Algorithm X so that it backtracks immediately after discovering a single unwanted solution.

All 777 minimal subsets were found in this manner, involving 15441 invocations of Algorithm X, but needing a total of only about 1.5 gigamems of computation. Altogether (1, 22, 200, 978, 2780, 4609, 4249, 1950, 373, 22) candidate pairs were examined in rounds (32, 31, ... , 23); and exactly (8, 154, 387, 206, 22) solutions were found of sizes (27, 26, 25, 24, 23). The lexicographically last 23-clue subset, which is illustrated below, turns out to be a fairly tough puzzle, with 220 nodes in its search tree.

(Let f(x_1,...,x_32) be the monotone Boolean function ‘[the solution is unique, given the clues with xj = 1]’. This problem essentially asks for f’s prime implicants.)

55. If only one of those nine appearances has been specified, the other eight can always be permuted into another solution. And the entire diagram can be partitioned into nine disjoint sets of nine, all with the same property, thus requiring at least 2 · 9 clues.

This argument proves that all 18-clue characterizations must have a very special form. The interesting solution above makes a particularly satisfying puzzle. (The author found it with the help of a SAT solver; see Section 7.2.2.2.)

The same argument shows that (28b) needs at least 18 clues. But this time the corresponding SAT instance is unsatisfiable. Moreover, any 19-clue solution must have three clues in just one critical group of nine; the associated SAT instance, which insists on having at least one clue in each of the 2043 subsets of at most 18 cells that can be rearranged into new solutions, also is unsatisfiable. (Proved in 177 Mμ.) But hurrah, the special structure does lead to 20-clue examples, like the one above.

(The constructions for (28b) apply also to (28c), via the isotopism in answer 43.)

56. (We assume that a decent sudoku problem has only one solution.) An example with 40 irredundant clues, shown here, was first discovered by Mladen Dobrichev in 2014, after examining a huge number of cases. (Incidentally, the solution to this problem has no automorphisms.) An example with 41 irredundant clues would be a big surprise.

57. There are only 2 · 3! · 3! · 3! · 3! = 2592 possibilities for each box. So we can set up an exact cover problem with 9 · 2592 options, each of which names a box, nine row-column pairs, three horizontal trios, and three vertical trios. We can assume by symmetry that there’s only one option for box 0, namely ‘b_0 r_01 c_01 r_04 c_14 r_07 c_27 r_18 c_08 r_12 c_12 r_15 c_25 r_26 c_06 r_29 c_19 r_23 c_23 h_147 h_258 h_369 v_168 v_249 v_357’. Furthermore row 0 can be restricted to 1472AB3CD, where {A, C} = {5, 6} and {B, D} = {8, 9}. That reduces the number of options to 16417; and Algorithm X quickly ((58+54)Mμ) finds 864 solutions.

Such solutions were first discovered by A. Thoen and A. van de Wetering; see Thoen’s book Sudoku Patterns (2019), §2.7. All 864 are isomorphic under sudoku-solution-preserving permutations of rows and columns. One of the nicest is

58. Use the standard 729 sudoku options (30); but also include queen items ‘a(i+j)k′ b(i+j)k′’ in option (i, j, k) when k ≤ 7. Furthermore, in order to avoid getting each solution 7! 2! = 10080 times, force row 0 by adding a new primary item ‘*’ and new secondary items ‘*j’ for 0 ≤ j < 9, together with 20 options ‘* *0:f(0, p, q) ... *8:f(8, p, q)’ for 0 ≤ p < q < 9, p + q < 9, where f(j, p, q) = (j = p? 8: j = q? 9: 1 + j − [j > p] − [j > q]). Include ‘*j:k’ in option (0, j, k). There are only four solutions, found in 3 Gμ, centrally symmetric and reducing under transposition to only two. (See Appendix E, and Thoen’s book Sudoku Patterns (2019), §3.4.)

59. When ps precede rs precede cs precede bs in X1, the tree sizes are 1105, 910, 122.

60. Using the options (30), items rik and cjk should be secondary when row i or column j contains fewer than 6 cells. The puzzles are fun to solve by hand; but in a pinch, Algorithm X will traverse search trees of sizes 23, 26, and 16 to find the answers:

[These are the first of 26 elegant puzzles announced by Serhiy and Peter Grabarchuk on Martin Gardner’s 100th birthday (21 October 2014) and posted at puzzlium.com.]

61. Exactly 1315 of the (255)=53130 ways to retain five clues result in a unique solution, and 175 of them involve all five digits. The lexicographically first is Fig. A–2(a).

62. Follow the hint; the undesired straight n-ominoes can be rejected easily in step R2 by examining vn−1 and v_0. This quickly produces (16, 105, 561, 2804, 13602) box options, for n = (3, 4, 5, 6, 7), which can be fed to Algorithm X to get jigsaw patterns.

There are no patterns for n = 3. But n = 4 has 33 patterns, which divide into eight equivalence classes under rotation and/or reflection:

(The number of symmetries is shown below each arrangement; notice that 8/1 + 8/1 + 8/2 + 8/2 + 8/2 + 8/4 + 8/4 + 8/8 = 33.) Similarly, n = 5 has 266 equivalence classes, representing 256 · (8/1) + 7 · (8/2) + 3 · (8/4) = 2082 total patterns; n = 6 has 40237 classes, representing 39791 · (8/1) + 439 · (8/2) + 7 · (8/4) = 320098 patterns in all.

The computation gets more serious in the case n = 7, when Algorithm X needs about 1.9 Tμ to generate the 132,418,528 jigsaw patterns. These patterns include 16,550,986 classes with no symmetry, and 2660 with one nontrivial symmetry. The latter break down into 2265 that are symmetric under 180° rotation, 354 that are symmetric under horizontal reflection, and 41 that are symmetric under diagonal reflection. Here are some typical symmetric examples:

(It’s not difficult to generate all of the symmetric solutions for slightly higher values of n; three of the classes for n = 8, shown above, have more than 2 symmetries. And the case n = 9 contains two patterns with 8-fold symmetry besides the standard sudoku boxes: See Fig. A–2(b) and (c), where the latter might be called windmill sudoku! For complete counts for n = 8 and n = 9, with straight n-ominoes allowed, see Bob Harris’s preprint “Counting nonomino tilings,” presented at G4G9 in 2010.)

Fig. A–2. Jigsaw sudoku patterns.

63. A simple modification of exercise 7.2.2–76 will generate the 3173 boxes that have the desired rainbow property. An exact cover problem, given those 3173 options, shows (after 1.2 Gμ of computation) that the boxes can be packed in 98556 ways. If we restrict the options to the 3164 that aren’t sudoku boxes, the number of packings goes down to 42669, of which 24533 are faultfree. Figure A–2(d) is a faultfree example.

64. (a) When n = 4, one of the eight classes in answer 62 (the 2nd) has no solutions; another (the 5th) is clueless. When n = 5, eight of the 266 classes have no solution; six are clueless. When n = 6, 1966 of 40237 are vacuous and 28 are clueless.

(Maxime’s original puzzle appeared in the newsletter of Chicago Area Mensa [ChiMe MM, 3 (March 2000), 15]. Algorithm X solves it with a 40-node search tree. But the tree size would have been 215 if he’d put ABCDEF in the next row down!)

(b) (Solution by Bob Harris, www.bumblebeagle.org/dusumoh/proof/, 2006.) The clueless jigsaw for n = 4 generalizes to all larger n, as illustrated here for n = 7: First a = 3; hence b = 3; ... ; hence f = 3. Then g = 4; hence h = 4; ... ; hence l = 4. And so on. Finally we know where to place the 2’s and the 1’s. (This proof shows that, for odd n > 3, there’s always an n × n jigsaw sudoku whose clues lie entirely on the main diagonal. Is there also a general construction that works for even values of n? An 8 × 8 example appears in exercise 65.)

65. (The author designed these puzzles with the aid of exercises 62 and 64. Similar puzzles have been contrived by J. Henle, Math. Intelligencer 38, 1 (2016), 76–77.)

66. (Puzzles like this might be too difficult for humans, but not for Algorithm C.) Extend the 729 options (30) by adding ‘ij:k’, where ij is a new secondary item for 0 ≤ i, j < 9. Also add eighteen new primary items k for 1 ≤ k ≤ 9 and sj for 0 ≤ j < 9, where k represents card k and sj represents a slot in the 3×3 array. Each item k has nine options, for the nine slots in which it might be placed; for example, the options for item 2 are ‘2 s_0 00:2 11:3 22:4 20:1’, ‘2 s_1 03:2 14:3 25:4 23:1’, ... , ‘2 s_8 66:2 77:3 88:4 86:1’.

There are 9! ways to place the cards in slots; but only 9!/(3! 3!) = 10080 are actually different, because the rows and columns can be permuted independently without changing the number of sudoku solutions. Suppose card cj goes into slot sj; then we can assume without loss of generality that c_0 = 1 and that c_4 = min(c_4, c_5, c_7, c_8). (To incorporate these constraints, give only one option for card 1 and only eight options for cards 2–9; use ordering tricks like (26) to ensure that c_4 < c_5, c_4 < c_7, c_4 < c_8.)

With this understanding, puzzle (i) has only one solution, and only when c_0 ... c_8 = 192435768. (That solution has six automorphisms, in the sense of exercise 114.) Puzzle (ii) has a unique solution when c_0 ... c_8 = 149523786. It also has ten sudoku solutions when the slot permutation is 149325687; so we can’t use that placement.

67. (a) (Solution by A. E. Brouwer, homepages.cwi.nl/~aeb/games/sudoku/nrc.xhtml, 2006.) The four new boxes force also aaaaaaaaa, ... , eeeeeeeee to be rainbows.

(b) Introduce new primary items byk′ for 0 ≤ y < 9 and 1 ≤ k ≤ 9. Add byk′ to option (30) with y = 3⌊iτ/3⌋ + ⌊jτ/3⌋, where τ is the permutation (03)(12)(58)(67).

(c) With items byk′ only considered for y ∈ {0, 2, 6, 8}, Algorithm X’s search tree grows from 77 nodes to 231 for (i), and from 151 nodes to 708 for (ii).

[Puzzle (ii) is a variant of an 11-clue example constructed by Brouwer. The minimum number of clues necessary for hypersudoku is unknown.]

(d) True. (That’s the permutation τ in (b), applied to both rows and columns.)

68. (a) A simple backtrack program generates all convex n-ominoes whose top cell(s) are in row 0 and whose leftmost cell(s) are in column 0. [This problem has respectively (1, 2, 6, 19, 59, 176, 502) solutions for 1 ≤ n ≤ 7; see M. Bousquet-Mélou and J.-M. Fédou, Discrete Math. 137 (1995), 53–75, for the generating function.] The resulting (1, 4, 22, 113, 523, 2196, 8438) placements into an n × n box yield exact cover problems as in answer 62. Considering symmetries, we find 1 · (8/4) = 2 patterns when n = 2; 1·(8/1)+1·(8/4) = 10 patterns when n = 3; 10·(8/1)+7·(8/2)+4·(8/4)+1·(8/8) = 117 when n = 4; 355·(8/1)+15·(8/2)+4·(8/4) = 2908 when n = 5; 20154·(8/1)+342·(8/2)+ 8·(8/4) = 162616 when n = 6; 2272821·(8/1)+1181·(8/2)+5·(8/4) = 18187302 when n = 7. (Exercise 62 had different results because it disallowed straight n-ominoes.)

(b) There are 325 such nonominoes touching row 0 and column 0, leading to 12097 placements and 1014148 · (8/1) + 119 · (8/2) + 24 · (8/4) + 1 · (8/8) = 8113709 patterns. If we exclude the 3 × 3 nonomino, and its 49 placements, the number of patterns goes down to 675797 · (8/1) = 5406376.

[Convex polyominoes were introduced by Klarner and Rivest; see answer 303.]

69. Say that an “Nk” is a suitable nonomino placement that has k Bs and 9 − k Ls. Only two cases give seven wins for B: 1 N_6, 6 N_5, 2 N_0; 7 N_5, 1 N_1, 1 N_0. With the given voting pattern there are respectively (1467, 2362, 163, 2) options for N_6, N_5, N_1, N_0. Algorithm M provides the desired multiplicities. After 12 Mμ of computation we find that there are no solutions in case 1 but 60 solutions in case 2, one of which is shown.

(Of course the author does not recommend secret deals such as this! The point is that unfair gerrymandering is easy to do and hard to detect. Indeed, a trial of 1000 random voter patterns, each with 5/4 split in the nine standard 3×3 districts, included 696 cases that could be gerrymandered to seven Big-Endian districts using only convex nonominoes that fit in a 5 × 5. Eight of those cases could also achieve a 4 × 4 fit.)

[Similar studies, using realistic data, go back to R. S. Garfinkel’s Ph.D. thesis Optimal Political Districting (Baltimore: Johns Hopkins University, 1968).]

70. In (a), four pieces change; in (b) the solution is unique:

Notice that the spot patterns , , and are rotated when a domino is placed vertically; these visual clues, which would disambiguate (a), don’t show up in the matrix.

(Dominosa was invented by O. S. Adler [Reichs Patent #71539 (1893); see his booklet Sperr-Domino und Dominosa (1912), 23–64, written with F. Jahn]. Similar “quadrille” problems had been studied earlier by E. Lucas and H. Delannoy. See Lucas’s Récréations Mathématiques 2 (1883), 52–63; W. E. Philpott, JRM 4 (1971), 229–243.

71. Define 28 vertices Dxy for 0 ≤ x ≤ y ≤ 6; 28 vertices ij for 0 ≤ i < 7, 0 ≤ j < 8, and i + j even; and 28 similar vertices ij with i + j odd. The matching problem has 49 triples of the form {Dxy, ij, i(j+1)} for 0 ≤ i, j < 7, as well as 48 of the form {Dxy, ij, (i+1)j} for 0 ≤ i < 6 and 0 ≤ j < 8, corresponding to potential horizontal or vertical placements. For example, the triples for exercise 70(a) are {D06, 00, 01}, {D56, 01, 02}, ... , {D23, 66, 67}; {D01, 00, 10}, {D46, 01, 11}, ... , {D12, 57, 67}.

72. Model (i) has M = 56!/8!7 ≈ 4.10 × 10^42 equally likely possibilities; model (ii) has N = 1292697 · 28! · 2^21 ≈ 8.27 × 10^41, because there are 1292697 ways to pack 28 dominoes in a 7 × 8 frame. (Algorithm X will quickly list them all.) The expected number of solutions per trial in model (i) is therefore N/M ≈ 0.201.

Ten thousand random trials with model (i) gave 216 cases with at least one solution, including 26 where the solution was unique. The total number ∑x of solutions was 2256; and ∑x^2 = 95918 indicated a heavy-tailed distribution whose empirical standard deviation is ≈ 3.1. The total running time was about 250 Mμ.

Ten thousand random trials with model (ii), using random choices from a precomputed list of 1292697 packings, gave 106 cases with a unique solution; one case had 2652 of them! Here ∑x = 508506 and ∑x^2 = 144119964 indicated an empirical mean of ≈ 51 solutions per trial, with standard deviation ≈ 109. Total time was about 650 Mμ.

73. From 66110144/26611514/52132140/55322200/53242006/36430565/33643054 we get 730,924 solutions, which is the current record. This array, found by Michael Keller in 2004, has the surprising property that every candidate placement, except for the ‘21’ in ‘521’, occurs in at least one solution. (In fact, in at least 31,370 solutions!)

74. One way to obtain candidate arrays is to formulate an MCC problem: Given one of the 1292697 matchings of answer 72, let there be options ‘Puv xy tu:x tv:y’, ‘Puv xy tu:y tv:x’ for uv in the matching, and ‘Quv Dxy tu:x tv:y’, ‘Quv Dxy tu:y tv:x’ for uv not in the matching; here 0 ≤ x ≤ y ≤ 6, and duplicate options are omitted when x = y. Give each Dxy multiplicity 3. Also add 28 further options ‘# Dxy’, where # has multiplicity 15 (because 15 pairs xy should have only two spurious appearances).

For fun, the author chose a tatami tiling for the matching (see exercise 7.1.4– 215), and obtained one candidate every 70 Mμ or so when the nonsharp variant of Algorithm M was applied with randomization as in exercise 31. Surprisingly, the first 10000 candidates yielded 2731 solutions, of which the hardest (with a 572-node search tree) was 15133034/21446115/22056105/65460423/22465553/61102332/63600044.

[See www.solitairelaboratory.com/puzzlelaboratory/DominoGG.xhtml.]

75. (a) (x ○ y) ○ x = (x ○ y) ○ (y ○ (x ○ y)) = y.

(b) All five are legitimate. (The last two are gropes because f(t + f(t)) = t for 0 ≤ t < 4 in each case; they are isomorphic if we interchange any two elements. The third is isomorphic to the second if we interchange 1 ↔ 2. There are 18 grope tables of order 4, of which (4, 12, 2) are isomorphic to the first, third, and last tables shown here.)

(c) For example, let x ○ y = (− x − y) mod n. (More generally, if G is any group and if α ∈ G satisfies α^2 = 1, we can let x ○ y = αx− αy−α. If G is commutative and α ∈ G is arbitrary, we can let x ○ y = x− y− α.)

(d) For each option of type (i) in an exact covering, define x ○ x = x; for each of type (ii), define x ○ x = y, x ○ y = y ○ x = x; for each of type (iii), define x ○ y = z, y ○ z = x, z ○ x = y. Conversely, every grope table yields an exact covering in this way.

(e) Such a grope covers n^2 items with k options of size 1, all other options of size 3. [F. E. Bennett proved, in Discrete Mathematics 24 (1978), 139–146, that such gropes exist for all k with 0 ≤ k ≤ n and k ≡ n^2 (modulo 3), except when k = n = 6.]

Notes: The identity x○(y○x) = y seems to have first been considered by E. Schröder in Math. Annalen 10 (1876), 289–317 [see ‘(C_0)’ on page 306], but he didn’t do much with it. In a class for sophomore mathematics majors at Caltech in 1968, the author defined gropes and asked the students to discover and prove as many theorems about them as they could, by analogy with the theory of groups. The idea was to “grope for results.” The official modern term for a grope is a real jawbreaker: semisymmetric quasigroup.

76. (a) Eliminate the n items for xx; use only the 2(n3) options of type (iii) for which y ≠ z. (Idempotent gropes are equivalent to “Mendelsohn triples,” which are families of n(n − 1)/3 three-cycles (xyz) that include every ordered pair of distinct elements. N. S. Mendelsohn proved [Computers in Number Theory (New York: Academic Press, 1971), 323–338] that such systems exist for all n ≢ 2 (modulo 3), except when n = 6.

(b) Use only the (n+12) items xy for 0 ≤ x ≤ y < n; replace options of type (ii) by ‘xx xy’ and ‘xy yy’ for 0 ≤ x < y < n; replace those of type (iii) by ‘xy xz yz’ for 0 ≤ x < y < z < n. (Such systems, Schröder’s ‘(C_1) and (C_2)’, are called totally symmetric quasigroups; see S. K. Stein, Trans. Amer. Math. Soc. 85 (1957), 228– 256, §8. If idempotent, they’re equivalent to Steiner triple systems.)

(c) Omit items for which x = 0 or y = 0. Use only the 2(n−13) options of type (iii) for 1 ≤ x < y, z < n and y ≠ z. (Indeed, such systems are equivalent to idempotent gropes on the elements {1,...,n − 1}.)

77. Use primary items v and v′ for each vertex of G and H; also secondary items ee′ for each pair of edges e and e′ in G and the complement of H. There are n^2 options, namely ‘v v′ ∪e(v),e′(v′) e(v),e′(v′)’, where e(v) ranges over all edges v — u in G and e′ (v′) ranges over all nonedges v′ ⌿ u′ in H. (The solutions to this problem are the one-to-one matchings v ↔ v′ of the vertices such that u — v implies u′ — v′.)

78. For example, CATALANDAUBOREL, GRAMARKOFFKNOPP, ABELWEIERSTRASS, BERTRAND-HERMITE, CANTORFROBENIUS, GLAISHERHURWITZ, HADAMARDHILBERT, HENSELKIRCHHOFF, JENSENSYLVESTER, MELLINSTIELTJES, NETTORUNGESTERN, MINKOWSKIPERRON.

79. In an n × n array for word search, every k-letter word generates (n + 1 − k) · n · 4 horizontal/vertical options and (n + 1 − k)2 · 4 diagonal options. So the desired answer is (2, 5, 6, 5, 3, 5, 0, 1) · (1296, 1144, 1000, 864, 736, 616, 504, 400) = 24320.

80. Item q is selected at level 0, trying option x_0 = 8, ‘q x y:A p’. We cover q, then cover x, then purify y to color A, and cover p; but at level 1 we find that item r’s list is empty. So we backtrack: Uncover p, unpurify y, uncover x — and try option x_0 = 20, ‘q x:A’, hence purifying x to color A. This time at level 1 we try x_1 = 12, ‘p r x:A y’. That causes us to cover p, then cover r, and then (since x is already purified) to cover y. At level 2 we discover that we’ve found a solution! Here’s what’s in memory:

i:

0

1

2

3

4

5

6

NAME(i):

—

p

q

r

x

y

—

LLINK(i):

0

0

1

0

6

4

4

RLINK(i):

0

3

3

0

6

6

4

x:

0

1

2

3

4

5

6

LEN(x), TOP(x):

—

1

2

1

2

0

0

ULINK(x):

—

12

20

23

18

5

—

DLINK(x):

—

12

8

23

14

5

10

COLOR(x):

—

—

—

—

—

—

0

x:

7

8

9

10

11

12

13

TOP(x):

1

2

4

5

−1

1

3

ULINK(x):

1

2

4

5

7

1

3

DLINK(x):

12

20

14

15

15

1

23

COLOR(x):

0

0

0

A

0

0

0

x:

14

15

16

17

18

19

20

TOP(x):

4

5 −2

1

4

−3

2

ULINK(x):

4

5

12

12

14

17

8

DLINK(x):

18

24

18

1

4

21

2

COLOR(x):

−1

0

0

0

B

0

0

x:

21

22

23

24

25

TOP(x):

4

−4

3

5

−5

ULINK(x):

18

20

3

5

23

DLINK(x):

4

24

3

5

—

COLOR(x):

A

0

0

B

0

81. Almost true, if TOP and COLOR are stored in the same octabyte (so that only one is charged to read both). The only difference is when processing the input, because Algorithm X has no COLOR fields to initialize but Algorithm C zeroes them out.

82. True; the LEN field of secondary items doesn’t affect the computation.

83. Before setting i ← TOP(x_0) in step C6 when l = 0, let node x be the spacer at the right of x_0’s option, and set j ← TOP(x − 1). If j > N_1 (that is, if x_0’s option ends with the secondary item j), and if COLOR(x − 1) = 0, cover(j).

84. Let CUTOFF (initially ∞) point to the spacer at the end of the best solution found so far. We’ll essentially remove all nodes > CUTOFF from further consideration.

Whenever a solution is found, let node PP be the spacer at the end of the option for which xk = max(x_0,...,xl−1). If PP ≠ CUTOFF, set CUTOFF ← PP, and for 0 ≤ k < l remove all nodes > CUTOFF from the list for TOP(xk). (It’s easy to do this because the list is sorted.) Minimax solutions follow the last change to CUTOFF.

Begin the subroutine ‘uncover′ (i)’ by removing all nodes > CUTOFF from item i’s list. After setting d ← DLINK(q) in unhide′ (p), set DLINK(q) ← d ← x if d > CUTOFF. Make the same modifications also to the subroutine ‘unpurify(p)’.

Subtle point: Suppose we’re uncovering item i and encounter an option ‘i j ... ’ that should be restored to the list of item j; and suppose that the original successor ‘j a ... ’ of that option for item j lies below the cutoff. We know that ‘j a ... ’ contains at least one primary item, and that every primary item was covered before we changed the cutoff. Hence ‘j a ... ’ was not restored, and we needn’t worry about removing it. We merely need to correct the DLINK, as stated above.

85. Now let CUTOFF be the spacer just before the best solution known. When resetting CUTOFF, backtrack to level k − 1, where xk maximizes {x_0,...,xl−1}.

86. The steps below also estimate the profile of the search tree. Running time is estimated in terms of “updates” and “cleansings.” The user specifies a random seed and a desired number of trials; the final estimates are the averages of the (unbiased) estimates from each trial. Here we specify only how to make a single trial.

In step C1, also set D ← 1.

In step C2, estimate that the search tree has D nodes at level l. If RLINK(0) = 0, also estimate that there are D solutions.

In step C3, let θ be the number of options in the list of the chosen item i. If θ = 0, estimate that there are 0 solutions, and go to C7.

At the end of step C4, let k be uniformly random in [0..θ − 1]; then set xl ← DLINK(xl), k times.

Just before setting l ← l + 1 at the end of step C5, suppose you’ve just done U updates and C cleansings. (An “update” occurs when ‘cover’ sets LLINK(r) or ‘hide’ sets ULINK(d). A “cleansing” occurs when ‘commit’ calls ‘purify’ or ‘purify’ sets COLOR(q) ← −1.) Estimate that level l does D(U′ + θ · U) updates and DC cleansings, where U′ is the number of updates just done in step C4. Then set D ← θ · D.

Step C6 now should do absolutely nothing. Steps C7 and C8 don’t change.

Upon termination, all data structures will have been returned to their original state, ready for another random trial. These steps will have estimated the number of nodes, updates, and cleansings at each level. Sum those estimates to get the total estimated number of nodes, updates, and cleansings.

87. Use 2n primary items ai, dj for the “across” and “down” words, together with n^2 secondary items ij for the individual cells. Also use W secondary items w, one for each legal word. The XCC problem has 2Wn options, namely ‘ai i1:c_1 ... in:cn c_1 ... cn’ and ‘dj 1j:c_1 ... nj:cn c_1 ... cn’ for 1 ≤ i, j ≤ n and each legal word c_1 ... cn. (See (110).)

We can avoid having both a solution and its transpose by introducing W further secondary items w@ and appending c_1 ... cn@ at the right of each option for a_1 and d_1. Then exercise 83’s variant of Algorithm C will never choose a word for d_1 that it has already tried for a_1. (Think about it.)

But this construction is not a win for “dancing links,” because it causes massive amounts of data to go in and out of the active structure. For example, with the five-letter words of WORDS(5757), it correctly finds all 323,264 of the double word squares, but its running time is 15 teramems! Much faster is to use the algorithm of exercise 7.2.2–28, which needs only 46 gigamems to discover all of the 1,787,056 unrestricted word squares; the double word squares are easily identified among those solutions.

88. One could do a binary search, trying varying values of W. But the best way is to use the construction of exercise 87 together with the minimax variant of Algorithm C in exercise 84. This works perfectly, when the options for most common words come first.

Indeed, this method finds the double square ‘BLAST|EARTH|ANGER|SCOPE|TENSE’ and proves it best in just 64 Gμ, almost as fast as the specialized method of exercise 7.2.2–28. (That square contains ARGON, the 1720th most common five-letter word, in its third column; the next-best squares use PEERS, which has rank 1800.)

89. The “minimax” method of exercise 88 finds the first five squares of

ISTOMAYAGENOTSHOWNONEOPENWESTSTARTTHREEROOFSASSETPEERSCHESTSLUSTREOBTAINARENASCIRCLEASSESSHERTZESOPERATEMIMICALACERATEGENETICENDMOSTRESENTS
in respectively 200 Kμ, 15 Mμ, 450 Mμ, 25 Gμ, 25.6 Tμ. It struggles to find the best 6 × 6, because too few words are cut off from the search; and it thrashes miserably with the 24 thousand 7-letter words, because those words yield only seven extremely esoteric solutions. For those lengths it’s best to cull the 2038753 and 14513 unrestricted word squares, which the method of exercise 7.2.2–28 finds in respectively 4.6 Tμ and 8.7 Tμ.

90. An XCC problem works nicely, as in answer 88: There are 2p primary items ai and di for the final words, and pn + W secondary items ij and w for the cells and potential words, where 0 ≤ i < p and 1 ≤ j ≤ n. The Wp options going across are ‘ai i1:c_1 i2:c_2 ... in:cn c_1 ... cn’. The Wp options going down are ‘di i1:c_1 ((i+1) mod p)2:c_2 ... ((i+n−1) mod p)n:cn c_1 ... cn’ for left-leaning stairs; ‘di i1:cn ((i+1) mod p)2:cn−1 ... ((i+n−1) mod p)n:c_1 c_1 ... cn’ for right-leaning stairs. The modification to Algorithm C in exercise 83 saves a factor of 2p; and the minimax modification in exercise 84 hones in quickly on optimum solutions.

There are no left word stairs for p = 1, since we need two distinct words. The left winners for 2 ≤ p ≤ 10 are: ‘WRITE|WHOLE’; ‘MAKES|LIVED|WAXES’; ‘THERE|SHARE| WHOLE|WHOSE’;‘STOOD|THANK|SHARE|SHIPS|STORE’;‘WHERE|SHEEP|SMALL|STILL|WHOLE| SHARE’;‘MAKES|BASED|TIRED|WORKS|LANDS|LIVES|GIVES’;‘WATER|MAKES|LOVED|GIVEN| LAKES|BASED|NOTES|TONES’; ‘WHERE|SHEET|STILL|SHALL|WHITE|SHAPE|STARS|WHOLE| SHORE’; ‘THERE|SHOES|SHIRT|STONE|SHOOK|START|WHILE|SHELL|STEEL|SHARP’. They all belong to WORDS(500), except that p = 8 needs WORDS(504) for NOTED.

The right winners have a bit more variety: ‘SPOTS’; ‘STALL|SPIES’; ‘STOOD|HOLES| LEAPS’; ‘MIXED|TEARS|SLEPT|SALAD’; ‘YEARS|STEAM|SALES|MARKS|DRIED’; ‘STEPS| SEALS|DRAWS|KNOTS|TRAPS|DROPS’; ‘TRIED|FEARS|SLIPS|SEAMS|DRAWS|ERECT|TEARS’; ‘YEARS|STOPS|HOOKS|FRIED|TEARS|SLANT|SWORD|SWEEP’; ‘START|SPEAR|SALES|TESTS| STEER|SPEAK|SKIES|SLEPT|SPORT’; ‘YEARS|STOCK|HORNS|FUELS|BEETS|SPEED|TEARS| PLANT|SWORD|SWEEP’. They belong to WORDS(1300) except when p is 2 or 3.

[Arrangements equivalent to left word stairs were introduced in America under the name “Flower Power” by Will Shortz in Classic Crossword Puzzles (Penny Press, February 1976), based on Italian puzzles called “Incroci Concentrici” in La Settimana Enigmistica. Shortly thereafter, in GAMES magazine and with p = 16, he called them “Petal Pushers,” usually based on six-letter words but occasionally going to seven. Left word stairs are much more common than the right-leaning variety, because the latter mix end-of-word with beginning-of-word letter statistics.]

91. Consider all “kernels” c_1 ...c_14 that can appear as illustrated, within a right word stair of 5-letter words. Such kernels arise for a given set of words only if there are letters x_1 ...x_12 such that x_3x_4x_5c_2c_3, c_4c_5c_6c_7c_8, c_9c_10c_11c_12x_6, c_13c_14x_7x_8x_9, x_1x_2x_5c_5c_9, c_1c_2c_6c_10c_13, c_3c_7c_11c_14x_10, and c_8c_12x_7x_11x_12 are all in the set. Thus it’s an easy matter to set up an XCC problem that will find the multiset of kernels, after which we can extract the set of distinct kernels.

x1x2c1x3x4x5c2c3c4c5c6c7c8c9c10c11c12x6c13c14x7x8x9x10x11x12

Construct the digraph whose arcs are the kernels, and whose vertices are the 9-tuples that arise when kernel c_1 ...c_14 is regarded as the transition
c_1c_2c_3c_4c_5c_6c_7c_9c_10 → c_3c_7c_8c_9c_10c_11c_12c_13c_14.

This transition contributes two words, c_4c_5c_6c_7c_8 and c_1c_2c_6c_10c_13, to the word stair. Indeed, right word stairs of period p are precisely the p-cycles in this digraph for which the 2p contributed words are distinct.

Now we can solve the problem, if the graph isn’t too big. For example, WORDS(1000) leads to a digraph with 180524 arcs and 96677 vertices. We’re interested only in the oriented cycles of this (very sparse) digraph; so we can reduce it drastically by looking only at the largest induced subgraph for which each vertex has positive in-degree and positive out-degree. (See exercise 7.1.4–234, where a similar reduction was made.) And wow: That subgraph has only 30 vertices and 34 arcs! So it is totally understandable, and we deduce quickly that the longest right word stair belonging to WORDS(1000) has p = 5. That word stair, which we found directly in answer 90, corresponds to the cycle

Click here to view code image

SEDYEARST → DRSSTEASA → SAMSALEMA → MESMARKDR → SKSDRIEYE → SEDYEARST.

A similar approach applies to left word stairs, but the kernel configurations are reflected left-to-right; transitions then contribute the words c_8c_7c_6c_5c_4 and c_1c_2c_6c_10c_13. The digraph from WORDS(500) turns out to have 136771 arcs and 74568 vertices; but this time 6280 vertices and 13677 arcs remain after reduction. Decomposition into strong components makes the task simpler, because every cycle belongs to a strong component. Still, we’re stuck with a giant component that has 6150 vertices and 12050 arcs.

The solution is to reduce the current subgraph repeatedly as follows: Find a vertex v of out-degree 1. Backtrack to discover a simple path, from v, that contributes only distinct words. If there is no such path (and there usually isn’t, and the search usually terminates quickly), remove v from the graph and reduce it again.

With this method one can rapidly show that an optimum left word stair from WORDS(500) has period length 36: ‘SHARE|SPENT|SPEED|WHEAT|THANK|CHILD|SHELL| SHORE|STORE|STOOD|CHART|GLORY|FLOWS|CLASS|NOISE|GAMES|TIMES|MOVES|BONES| WAVES|GASES|FIXED|TIRED|FEELS|WALLS|WORLD|ROOMS|WORDS|DOORS|PARTY|WANTS| WHICH|WHERE|SHOES|STILL|STATE’, with 36 other words that go down. Incidentally, GLORY and FLOWS have ranks 496 and 498, so they just barely made it into WORDS(500).

Larger values of W are likely to lead to quite long cycles from WORDS(W ). Their discovery won’t be easy, but the search will no doubt be instructive.

92. Use 3p primary items ai, bi, di for the final words; pn +2W secondary items ij, w, w@ for the cells and potential words, with 0 ≤ i < p and 1 ≤ j ≤ n (somewhat as in answer 90). The Wp options going across are ‘ai i1:c_1 i2:c_2 ... in:cn c_1 ...cn c_1 ...cn@’. The 2Wp options going down in each way are ‘bi i1:c_1 ((i+1) mod p)2:c_2... ((i+n−1) mod p)n:cn c_1 ...cn’ and ‘di i1:cn ((i+1) mod p)2:cn−1 ... ((i+n−1) mod p)n:c_1 c_1 ...cn’. The items w@ at the right of the ai options save us a factor of p.

Use Algorithm C (modified). We can’t have p = 1. Then comes ‘SPEND|SPIES’; ‘WAVES |LINED |LEPER’; ‘LOOPS |POUTS |TROTS |TOONS’; ‘SPOOL |STROP |STAID |SNORT | SNOOT’;‘DIMES|MULES|RIPER|SIRED|AIDED|FINED’;‘MILES|LINTS|CARES|LAMED|PIPED| SANER|LIVER’;‘SUPER|ROVED|TILED|LICIT|CODED|ROPED|TIMED|DOMED’;‘FORTH|LURES| MIRES|POLLS|SLATS|SPOTS|SOAPS|PLOTS|LOOTS’; ‘TIMES|FUROR|RUNES|MIMED|CAPED| PACED|LAVER|FINES|LIMED|MIRES’. (Lengthy computations were needed for p ≥ 8.)

93. Now p ≤ 2 is impossible. A construction like the previous one allows us again to save a factor of p. (There’s also top/bottom symmetry, but it is somewhat harder to exploit.) Examples are relatively easy to find, and the winners are ‘MILES|GALLS| BULLS’; ‘FIRES|PONDS|WALKS|LOCKS’; ‘LIVES|FIRED|DIKES|WAVED|TIRES’; ‘BIRDS| MARKS|POLES|WAVES|WINES|FONTS’; ‘LIKED|WARES|MINES|WINDS|MALES|LOVES|FIVES’; ‘WAXES|SITES|MINED|BOXES|CAVES|TALES|WIRED|MALES’; ‘CENTS|HOLDS|BOILS|BALLS| MALES|WINES|FINDS|LORDS|CARES’; ‘LOOKS|ROADS|BEATS|BEADS|HOLDS|COOLS|FOLKS| WINES|GASES|BOLTS’. [Such patterns were introduced by Harry Mathews in 1975, who gave the four-letter example ‘TINE|SALE|MALE|VINE’. See H. Mathews and A. Brotchie, Oulipo Compendium (London: Atlas, 1998), 180–181.]

94. Set up an XCC problem with primary items k, pk, and secondary items xk, for 0 ≤ k < 16, and with options ‘j pk xk:a x(k+1) mod 16 :b x(k+3) mod 16 :c x(k+4) mod 16 :d’ for 0 ≤ j, k < 16, where j = (abcd)2. The solution (0000011010111011) is essentially unique (except for cyclic permutation, reflection, and complementation). [See C. Flye Sainte-Marie, L’Intermédiaire des Mathématiciens 3 (1896), 155–161.]

95. Use 2m primary items ak, bk, and m secondary items xk, for 0 ≤ k < m. Define m^2 options of size 2 + n, namely ‘aj bk xj:t_1 x(j+1) mod m:t_2 ... x(j+n−1) mod m:tn’, where t_1t_2 ...tn is the kth binary vector of interest. However, save a factor of m by omitting the options with j = 0 and k > 0, and the options with j > 0 and k = 0.

The case (7, 0, 3) has 137216 solutions, found in 8.5 gigamems; the case (7, 3, 4) has 41280 solutions, found in 3.2 gigamems. (We can make the items bk secondary instead of primary. This makes the search tree a bit larger. But it actually saves a little time, because the MRV heuristic causes branching on aj and maintains a good focus; less time is spent computing that heuristic when bk isn’t primary. Alternatively we could make the items ak secondary (or even omit them entirely, which would have the same effect). But that would be a disaster! For example, the running time for case (7, 0, 3) would then increase to nearly 50 teramems, because focus is lost.)

Section 7.2.1 discusses other “universal cycles,” which can be handled similarly.

96. In fact, there are 80 solutions for which the bottom four rows are the complements of the top four. (This problem extends the idea of “ourotoruses” in exercise 7.2.1.1–109. One can also consider windows that aren’t rectangles. For example, the thirty-two ways to fill a cross of five cells can be identified with 32 positions of the generalized torus whose offsets are (4, ±4); see exercise 7–137.)

00000110
00010111
11001010
10001110
11111001
11101000
00110101
01110001

97. Use primary items jk, pjk, and secondary items dj,k, for 0 ≤ j < 3 and 0 ≤ k < 9, with the following three options for each 0 ≤ i, j < 3 and 0 ≤ k, k′ < 9: ‘jk pj′ k′ dj′, k′ :i dj′, k′+1: (i + a) dj′,k′ :(i + b) dj′+1,k′ +1:(i + c)’, for 0 ≤ j′ ≤ 1, and ‘jk p2 k′ d2, k′ :i d2,k′+1:(i+a) d0,k′−3:(i+b−1) d0,k′ −2:(i+c−1)’, where 9j +k = (abc)3; sums involving i are mod 3, while sums involving k′ are mod 9. We can assume that 00 is paired with p_00. Then there are 2 · 2898 = 5796 solutions D; all have D ≠ DT .

98. Given a 3SAT problem with clauses (li_1 ∨ li_2 ∨ li_3) for 1 ≤ i ≤ m, with each lij∈{x1,x¯1,.⁢ .⁢ .⁢ ,xn,x¯n}, construct an XCC problem with 3m primary items ij (1 ≤ i ≤ m, 1 ≤ j ≤ 3) and n secondary items xk (1 ≤ k ≤ n), having the following options: (i) ‘li_1 li_2’, ‘li_2 li_3’, ‘li_3 li_1’; (ii) ‘lij xk:1’ if lij = xk, ‘lij xk:0’ if lij=x¯k. That problem has a solution if and only if the given clauses are satisfiable.

99. True — but perhaps with many more secondary items and much longer options: Let x be a secondary item to which a color has been assigned, in some XCC problem A; and let O be the options in which x appears. Replace A by a new problem A′, by deleting item x and adding new secondary items x{o,p} for each o, p ∈ O for which x gets different colors in A. And for each o ∈ O, replace item x in o by the set of all x{o,p} that apply. If A′ still involves colors, replace it by A′ in a similar way, until all colors disappear.

100. (a) There are five solutions: 00112, 00122, 01112, 01122, 11111.

(b) Let there be five primary items, {#1, #2, #3, #4, #5}, and five secondary items, {x_1,x_2,x_3,x_4,x_5}. Item #1 enforces the binary constraint x_1 ≤ x_2, and has the options ‘#1 x_1:0 x_2:0’; ‘#1 x_1:0 x_2:1’; ‘#1 x_1:0 x_2:2’; ‘#1 x_1:1 x_2:1’; ‘#1 x_1:1 x_2:2’; ‘#1 x_1:2 x_2:2’. Similar options for #2, #3, and #4 will enforce the constraints x_2 ≤ x_3, x_3 ≤ x_4, and x_4 ≤ x_5. Finally, the options ‘#5 x_1:0 x_3:1 x_5:2’; ‘#5 x_1:0 x_3:2 x_5:1’; ‘#5 x_1:1 x_3:0 x_5:2’; ‘#5 x_1:1 x_3:1 x_5:1’; ‘#5 x_1:1 x_3:2 x_5:0’; ‘#5 x_1:2 x_3:0 x_5:1’; ‘#5 x_1:2 x_3:1 x_5:0’ will enforce the ternary constraint x_1 + x_3 + x_5 = 3.

(c) Use primary items #j for 1 ≤ j ≤ m, one for each constraint, and secondary items xk for 1 ≤ k ≤ n, one for each variable. If constraint Cj involves the d variables xi_1 , ... , xid, include options ‘#j xi_1 :a_1 ... xid :ad’ for each legal d-tuple (a_1, . . . ,ad).

(Of course this construction isn’t efficient for all instances of CSP; furthermore, we can often find substantially better ways to encode a particular CSP as an XCC instance, because this method uses only one primary item in each option. But the idea that underlies this construction is a useful mental tool when formulating particular problems.)

101. Notice that the final sentence implies two further clues:

- Somebody trains a zebra.

- Somebody prefers to drink just plain water.

Let there be primary items #k for 1 ≤ k ≤ 16, one for each clue. And let the 5 · 5 secondary items Nj, Jj, Pj, Dj, Cj represent the nationality, job, pet, drink, and color associated with house j, for 0 ≤ j < 5. There are respectively (5, 5, 5, 5, 1, 5, 1, 5, 4, 5, 8, 5, 8, 8, 5, 5) options for clues (1, ... , 16), typified by ‘#1 Nj:England Cj:red’, for 0 ≤ j < 5; ‘#5 N_0:Norway’; ‘#9 Ci:white Ci+1:green’, for 0 ≤ i < 4; ‘#14 Ji:nurse Pi+1:fox’, ‘#14 Pi:fox Ji+1:nurse’, for 0 ≤ i < 4; ‘#15 Pj:zebra’, for 0 ≤ j < 5.

A more complex formulation enforces the redundant “all-different” constraint by introducing 5 · 5 additional secondary items to represent the inverses of Nj, Jj, Pj, Dj, Cj. For example, the options for #1 then become ‘#1 Nj:England NEngland−: j Cj:red Cred−:j’. (With those additional items, Algorithm C will infer C_1:blue immediately from #5 and #11; but without them, #5 doesn’t immediately make N_1:Norway illegal. They reduce the search tree size from 112 to 32 nodes. However, the time they save during the search just barely compensates for the extra time that they consume in step C1.)

The inverses alone are not sufficient; they don’t forbid, say, NEngland−=NJapan−.

[The author of this now-famous puzzle is unknown. Its first known publication, in Life International 35 (17 December 1962), 95, used cigarettes instead of occupations.]

102. As in answer 7.2.2–68, let’s find all stable extensions of a given partially labeled digraph. And let’s allow sinks too; we can assume that every vertex with out-degree d ≤ 1 is labeled d. The following XCC formulation is based on ideas of R. Bittencourt.

Let Δ be the maximum out-degree. Introduce primary items Hv, Iv, Evd, and secondary items v, hvd, ivd, for 0 ≤ d ≤ Δ and all vertices v. The color of v will be λ(v), the label of v; the color of hvd will denote the Boolean quantity ‘[v sees d]’, meaning that λ(w) = d for some w with v → w; and the color of ivd will denote ‘[λ(v) = d]’. The options for Hv are ‘Hv v:d ∪k=0Δ{hvk:ek}’ where e_0 + ··· + eΔ = d. The options for Iv are Iv u:d ∪k=0Δ{ivk:[k=d]} ∪u→v{hu⁢  d:1}’. And the options for Evd are ‘Evd hvd:1 iwk d:1 ∪j=1k−1{iw⁢j⁢ d:0}’ for 1 ≤ k ≤ d+(v) and ‘Evd⁢  hv⁢d:0∪j=1d+(v){iw⁢j⁢ d:0}’, when v → w_1, ... , v → wd+(v). For example, if the vertices of the puzzle in exercise 7.2.2–68 are named 00, ... , 99, some of the options of its unique solution are ‘H_00 00:3 h_000:0 h_001:0 h_002:0 h_003:0 h_004:1 h_005:0 h_006:0 h_007:1 h_008:0 h_009:1’; ‘I_00 00:3 i_000:0 ...i_002:0 i_003:1 i_004:0 ...i_009:0 h_013:1 h_033:1 h_053:1 h_703:1 h_803:1’; ‘E_004 h_004:1 i_104:0 ...i_404:0 i_504:1’.

Of course many of those options can be greatly simplified, because many of the quantities are known from the given labels. We know the color of ivd when λ(v) is given; we know the color of hvd when v sees d in the given puzzle. We don’t need Iv when v is labeled; we don’t even need Evd, when v is known to see d. If v has out-degree d and already sees some label twice, we know that ivd is 0. And so on. In the pi day puzzle such simplifications reduce 60 thousand options on 1200 + 1831 items to 11351 options on 880+1216 items. That’s still a lot, and Algorithm C needs 135 Mμ to input them; but then it finds the solution and proves it unique after 25 more Mμ. (The highly tuned method of answer 7.2.2–68 needed only 7 Mμ to prove uniqueness. But that method solves only a small class of problems that happen to reduce nicely.)

Bittencourt notes that further speedup is possible when two arrows point in the same direction. (This happens 123 times in the pi day puzzle.) In general if v → w implies u → w, we must have λ(u) ≥ λ(v); and this condition can be enforced by introducing a new primary variable whose options allow u and v to have only appropriate combinations of colors.

103. (a) An all-interval row always has xn−1 = (x_0 + 1 + ··· + (n − 1)) mod n = (x_0 + n(n−1)/2) mod n = (x_0 +[n even] n/2) mod n.

(b) Let j, pj, dk, qk be primary items and let xj be a secondary item, for 0 ≤ j < n and 1 ≤ k < n. There’s an option ‘j pt xj:t’ for 0 ≤ j, t < n, omitted when (j = 0 and t = 0) or (j = n − 1 and t = n/2). And there’s an option ‘dk qt xt−1:i xt:(i + k) mod n’ for 1 ≤ k, t < n and 0 ≤ i < n. Then the tone row and its intervals are permutations.

There are (1, 2, 4, 24, 288, 3856) solutions for n = (2, 4, 6, 8, 10, 12). [These values were first computed by D. H. Lehmer, Proc. Canadian Math. Congress 4 (1959), 171–173, for n = 12 and E. N. Gilbert SIAM Review 7 (1965), 189–198, for n < 12.]

For larger n, Algorithm C is not at all competitive with a straightforward backtrack algorithm, which uses Algorithm 7.2.1.2X to find all suitable permutations of the n−1 intervals: That method needs only 100 Mμ to find all 89328 solutions when n = 14, compared to 107 Gμ by Algorithm C! With backtracking we can generate all 2755968 solutions for n = 16 in 4.7 Gμ, and all 103653120 solutions for n = 18 in 281 Gμ.

(c) The intervals between adjacent classes in xQ are the same as those of x, except that xk − xk−1 is replaced by x_0 − xn−1. And we know that x_0 − xn−1 = ±n/2.

(d) True; both are xk−1 ...x_0xn−1 ...xk. (Also (cx)R = c(xR); (cx)Q = c(xQ).)

(e) The solution for n = 2 has every possible symmetry; and both solutions x for n = 4 are equivalent to xR, −xQ, and −xQR. But for n > 4 one can show that x is equivalent to at most one of the 4ϕ(n) rows cx, cxR, cxQ, cxQR besides itself. We obviously can’t have x ≡ cx when c = 1. An elementary but nontrivial proof shows also that x ≡ cxR implies c mod n = 1; x ≡ cxQ implies c mod n = n−1; x ≡ cxQR implies c mod n = n/2 + 1 and n mod 8 = 4. (See Richard Stong in AMM, to appear.) Gilbert stated incorrectly [page 196] that no solutions of the latter kind exist; he had overlooked 12-tone rows such as 0 3 9 1 2 4 11 8 7 5 10 6, 0 1 4 9 3 11 10 8 5 7 2 6, 0 1 8 11 10 3 9 5 7 4 2 6, for which x ≡ 7xQR. Similarly, the 20-tone row 0 1 3 11 2 19 13 9 12 7 14 18 4 17 16 8 6 15 5 10 satisfies x ≡ 11xQR.)

At any rate, the transformations of (c) partition the solutions into clusters of size 2ϕ(n) when there’s symmetry, 4ϕ(n) when there’s not. Gilbert enumerated the cases of symmetry when n < 12; R. Morris and D. Starr did it when n = 12[J. Music Theory 18 (1974), 364–389]. For n = (6, 8, 10, 12, 14, 16, 18) the number of clusters with x ≡ xR turns out to be respectively (1, 1, 6, 22, 48, 232, 1872); the number of clusters with x ≡ −xQ turns out to be (0, 0, 2, 15, 0, 0, 1346); also n = 12 has 15 cases with x ≡ 7xQR.

104. (a) We may assume that x_0 = 0. There’s a constant cr such that ykr ≡ xk−1 + cr (modulo n) for 1 ≤ k ≤ n. Thus yr = xr−1 ≡ cr; yr^2 = x(r^2−1) mod p ≡ xr−1 + cr ≡ 2cr; yr^3 = x(r^3−1) mod p ≡ x(r^2−1) mod p + cr ≡ 3cr; etc. Let r be primitive modulo p, so that {r mod p,...,rn mod p} = {1,..., p − 1}, and let R = rd where crd mod n = 1. Then we’ve proved Rx(rk−1) mod p ≡ (rk mod p) (modulo p) for 1 ≤ k ≤ n; that is, Rxk−1 ≡ k.

Now suppose xk − xk−1 ≡ xl − xl−1 (modulo n). Then Rxk Rxl−1 ≡ Rxk−1 Rxl (modulo p); consequently (k +1)l ≡ k(l + 1) (modulo p), hence k = l.

(b) x(n) = xR. [See the papers by Lehmer and Gilbert in answer 103.]

105. There are just five solutions; the latter two are flawed by being disconnected:

Historical notes: The earliest known word search puzzle was “Viajando” by Henrique Ramos of Brazil, published in Almanaque de Seleções Recreativas (1966), page 43. Such puzzles were independently invented in America by Norman E. Gibat (1968). Jo Ouellet of Canada developed “Wonderword,” which puts the unused letters to use, in 1970.

106. When Algorithm C is generalized to allow non-unit item sums as in Algorithm M, it needs just 24 megamems to prove that there are exactly eight solutions — which all are rotations of the two shown here.

107. To pack w given words, use primary items {Pij, Ric, Cic, Bic, #k | 1 ≤ i, j ≤ 9, 1 ≤ k ≤ w, c ∈ {A, C, E, M, O, P, R, T, U}} and secondary items {ij | 1 ≤ i, j ≤ 9}. There are 729 options ‘Pij Ric Cjc Bbc ij:c’, where b = 3 ⌊(i − 1)/3⌋ + ⌈j/3⌉, together with an option ‘#k i_1j_1:c_1 ... iljl:cl’ for each placement of an l-letter word c_1 ...cl into cells (i_1,j_1), ... , (il,jl). Furthermore, it’s important to use the sharp preference heuristic (exercise 10) in step C3 of the algorithm.

A brief run then establishes that COMPUTER and CORPORATE cannot both be packed. But all of the words except CORPORATE do fit together; the (unique) solution shown is found after only 7.3 megamems, most of which are needed simply to input the problem. [This exercise was inspired by a puzzle in Sudoku Masterpieces (2010) by Huang and Snyder.]

108. (a, b) The author’s best solutions, thought to be minimal (but there is no proof), are below. In both cases, and in Fig. 71, an interactive method was used: After the longest words were placed strategically by hand, Algorithm C packed the others nicely.

[Solution (b) applies an idea by which Leonard Gordon was able to pack the names of presidents 1–42 with one less column. See A. Ross Eckler, Word Ways 27 (1994), 147; see also page 252, where OBAMA miraculously fits into Gordon’s 15 × 15 solution!]

109. To pack w given words, use w + m(n − 1) + (m − 1)n primary items {#k | 1 ≤ k ≤ w} and {Hij, Vij | 1 ≤ i ≤ m, 1 ≤ j ≤ n}, but with Hin and Vmj omitted; Hij represents the edge between cells (i, j) and (i, j + 1), and Vij is similar. There also are 2mn secondary items {ij, ij′ | 1 ≤ i ≤ m, 1 ≤ j ≤ n}. Each horizontal placement of the kth word c_1 ...cl into cells (i, j + 1), ... , (i, j + l) generates the option

‘#k ij:. ij′:0 i(j+1):c_1 i(j+1)′:1 Hi(j+1) i(j+2):c_2 i(j+2)′:1 Hi(j+2) ...

Hi(j+l−1) i(j+l):cl i(j+l)′:1 i(j+l+1):. i(j+l+1)′:0’

with 3l + 4 items, except that ‘ij:. ij′:0’ is omitted when j = 0 and ‘i(j+l+1):. i(j+l+1)′:0’ is omitted when j +l = n. Each vertical placement is similar. For example,

‘#1⁢  11:Z  11':1 V11 21:E  21':1  V21  31:R  31':1  V31  41:0  41':1  51:.  51':0’(*)

is the first vertical placement option for ZERO, if ZERO is word #1. When m = n, however, we save a factor of 2 by omitting all of the vertical placements of word #1.

To enforce the tricky condition (ii), we also include 3m(n−1)+3(m−1)n options:

‘Hij ij′:0 i(j+1)′:1 ij:.’

‘Vij ij′:0 (i+1)j′:1 ij:.’

‘Hij ij′:1 i(j+1)′:0 i(j+1):.’

‘Vij ij′:1 (i+1)j′:0 (i+1)j:.’

‘Hij ij′:0 i(j+1)′:0 ij:. i(j+1):.’

‘Vij ij′:0 (i+1)j′:0 ij:. (i+1)j:.’

This construction works nicely because each edge must encounter either a word that crosses it or a space that touches it. (Beware of a slight glitch: A valid solution to the puzzle might have several compatible choices for Hij and Vij in “blank” regions.)

Important: As in answer 107, the sharp preference heuristic should be used here, because it gives an enormous speedup.

The XCC problem for our 11-word example has 1192 options, 123 + 128 items, and 9127 solutions, found in 29 Gμ. But only 20 of those solutions are connected; and they yield only the three distinct word placements below. A slightly smaller rectangle, 7 × 9, also has three valid placements. The smallest rectangle that admits a solution to (i) and (ii) is 5 × 11; that placement is unique, but it has two components:

Suppose there are w words of total length s. Aaron Windsor suggests adding options ‘E ij:. ij′:0’ for 1 ≤ i ≤ m and 1 ≤ j ≤ n, where E is a new primary item representing an empty cell. All solutions to the MCC problem with the number of E’s in the interval [mn − s + w − 1 ..mn] are then either connected or contain a cycle.

Instead of generating all solutions to (i) and (ii) and discarding the disconnected ones, there’s a much faster way to guarantee connectedness throughout the search; but it requires major modifications to Algorithm C. Whenever no H or V is forced, we can list all active options that are connected to word #1 and not smaller than choices that could have been made earlier. Then we branch on them, instead of branching on an item. For example, if (∗) above is used to place ZERO, it will force H00 and H20 and V30. The next decision will be to place either EIGHT or ONE, in the places where they overlap ZERO. (However, we’ll be better off if we order the words by decreasing length, so that for instance #1 is EIGHT and #11 is ONE.) Interested readers are encouraged to work out the instructive details. This method needs only 630 Mμ to solve the example problem, as it homes right in on the three connected solutions.

110. Gary McDonald found this remarkable 20 × 20 solution in 2017:

A 19 × 19 is surely impossible, although no proof is known. L. Gordon had previously fit the names of presidents 1–42 into an 18 × 22 rectangle [Word Ways 27 (1994), 63].

111. (a) Set up an XCC problem as in answer 109, but with just three words AAA, AAAA, AAAAA; then adjust the multiplicities and apply Algorithm M. The two essentially distinct answers are shown below; one of them is disconnected, hence disqualified.

(b) Similarly, we find four essentially different answers, only two of which are OK:

Algorithm M handles case (b) with ease (5 Gμ). But it does not explore the space of possibilities for case (a) intelligently, and costs 591 Gμ.

112. (a) Yes: IMMATURE, MATURING, COMMUTER, GROUPING, TROUPING, AUTHORING, and THRUMMING. A straightforward backtracking program will quickly determine the presence or absence of any given string of letters.

(b) Let’s put DANCING and LINKS in there too. Then we obtain an array with 24 words from WORDS(5757) (like LOVER, ROSIN, SALVO, TOADS, TROVE); also ASKING, DOSING, LOSING, ORDAIN, SAILOR, SIGNAL, SILVER; also LANCING, LOANING, SOAKING, and even ORTOLAN. (Notice that TORTO occurs in two ways.)

To find such arrays, as suggested by R. Bittencourt, we can let word k be c_0 ...ct−1, and introduce primary items Wkl for 1 ≤ l < t to represent the placement of cl−1cl. Let Xu be a secondary item, for each cell u of the array, to be colored with some letter. Represent the king path for word k by giving color u to Pkl and color l to Qku when cl is in cell u, where Pkl and Qku are additional secondary items. There also are secondary items Dkv, for each internal vertex v. For example, if the cells and vertices are numbered rowwise, two of the options chosen for DANCING and LINKS in the example above are ‘W_03 X_3:N X_0:C P_02:3 P_03:0 Q_03:2 Q_00:3’ and ‘W_12 X_4:I X_2:N P_11:4 P_12:2 Q_14:1 Q_12:2 D_11’. The ‘D_11’ in the latter will prevent another step of word 1 between cells 2 and 4.

We can save a factor of nearly 4 by restricting the placement of, say, cl−1cl in word 0 when l = ⌊t_0/2⌋, so that cl−1 lies in the upper left quadrant and cl isn’t in the rightmost column. Then W_0l has only 26 options instead of the usual 94.

It turns out that exactly 10 essentially different Torto arrays contain DANCING, LINKS, TORTO, WORDS, and SOLVER; exactly 1444 contain THE, ART, OF, COMPUTER, and PROGRAMMING. They’re found by Algorithm C in 713 Gμ and 126 Gμ, respectively.

(c) Yes, in 140 ways (but we can’t add ELEVEN). Similarly, we can pack ZERO, ONE, ... , up to EIGHT, in 553 ways. And FIRST, ... , SIXTH can be packed in 72853 ways, sometimes without using more than 16 of the 18 cells. (These computations took (16, 5, 1.5) Tμ. Interesting words lurk in these arrays — can you spot them? See Appendix E.)

[The name ‘Torto’ was trademarked by Coquetel/Ediouro of Rio de Janeiro in 1977, and an example appeared in issue #1 of Coquetel Total magazine that year. Monthly puzzles still appear regularly in Coquetel’s magazine Desafio Cérebro. Bitten-court posed the problem of constructing Torto arrays from a given list of required words in 2011; see blog.ricbit.com/2011/05/torto-reverso.xhtml.]

113. First, we can find all sets of six or fewer letters that could be on such a block, by solving an MCC problem with 25 primary items TREES, ... , DEQUE of multiplicity [1 .. 26] and one primary item # of multiplicity [0 .. 6]. There are 22 options, ‘# ABOVE AVAIL GRAPH STACK TABLE VALUE’ through ‘# EMPTY’, one for each potential letter (listing all words that include that letter). This covering problem has 3223 solutions, found in 4 Mμ and ranging alphabetically from {A, B, C, D, E, I} to {E, L, R, T} to {L, N, R, T, U, V}.

Then we set up an XCC problem with 25 primary items TREES, ... , DEQUE and five primary items 1, ... , 5, together with 5 · 22 secondary items Aj, ... , Yj for 1 ≤ j ≤ 5. Each word has an option for each permutation of its letters (see Algorithm 7.2.1.2L), showing which letters it needs for that permutation of the blocks. (For example, QUEUE will have 30 options, beginning with ‘QUEUE E_1:1 E_2:1 Q_3:1 U_4:1 U_5:1’, which means that block 1 should have an E, ... , block 5 should have a U.) Break symmetry by giving only one of the 120 options for one of the words (FIRST, for example). Each of the 3223 potential sets of letters has five options of size 23, showing exactly which letters are present if block j uses that set; for example, the five options for {A, B, C, D, E, I} are ‘j Aj:1 ... Ej:1 Fj:0 ... Ij:1 ... Yj:0’ for 1 ≤ j ≤ 5. There are 18486 options altogether, of total length 403357; Algorithm C solves them in 225 Gμ.

For these words the five blocks must be {E, F, G, L, O, S}, {C, E, T, R, U, Y}, {A, L, M, N, Q, R}, {A, B, E, P, S, T}, {D, H, K, T, U, V}. (The XCC problem actually has 8 solutions, because TIMES, TREES, and VALUE can each be formed in two ways from those blocks.)

[This exercise is based on an idea by E. Riekstiņš, who realized that a classic puzzle called Castawords could be extended to words of length 5.]

114. Besides the primary items pij, rik, cjk, bxk of (30), introduce Rik, Cjk, and Bxk for the permuted array, as well as uk and vl to define a permutation. Also introduce secondary items πk to record the permutation and ij to record the value at cell (i, j). The permutation is defined by 81 options ‘uk vl πk:l’ for 1 ≤ k, l ≤ 9. And there are 9^4 = 6561 other options, one for each cell (i, j) of the board and each pair (k, l) of values before and after α is applied. If (ij)α = i′j′, let x′ = 3 ⌊i′/3⌋ + ⌊j′/3⌋. Then option (i, j, k, l) is normally ‘pij rik cjk bxk Ri′l Cj′l Bx′l ij:k i′j′:l πk:l’. However, if i′ = i and j′ = j, that option is shortened to ‘pij rik cjk bxk Ril Cjl Bxl ij:k πk:l’; and it is omitted when i = i′, j = j′, k ≠ l. The options (0,j,k,l) are also omitted when k ≠ j + 1, in order to force ‘123456789’ on the top row.

With that top row and with α = transposition, Algorithm C produces 30,258,432 solutions in 2.2 teramems. (These solutions were first enumerated in 2005 by E. Russell; see www.afjarvis.staff.shef.ac.uk/sudoku/sudgroup.xhtml.)

115. A similar method applies, but with additional items by⁢ k′ and By′l′ as in answer 67(b). The number of solutions is (a) 7784; (b) 16384; (c) 372; (d) 32. Here are examples of (a) and (d); the latter is shown with labels {0,..., 7, *}, to clarify its structure. [Enumerations (a), (b), (c) were first carried out by Bastian Michel in 2007.]

(a)

(d)

116. (a) Any triangle in μ(G) must be in G, because .

(b) Suppose μ(G) can be c-colored with some coloring function α, where α(w) = c. If α(v) = c for any v ∈ V, change it to α(v′). This gives a (c − 1)-coloring of G. [Hence a triangle-free graph on n vertices can have chromatic number Ω(log n). One can show nonconstructively that the triangle-free chromatic number can actually be Ω(n/log n)1/2; but currently known methods of explicit construction for large n achieve only Ω(n1/3). See N. Alon, Electronic J. Combinatorics 1 (1994), #R12, 1–8.]

(c) If G is χ-critical, so is μ(G): Let e ∈ E and suppose α is a (c − 1)-coloring of G \ e. Then we get c-colorings of all but one edge of μ(G) in several ways: (i) Set α(v′) ← c for all v ∈ V , and α(w) ← 1. (ii) Let u ∈ e, and set α(u) ← α(w) ← c; also set α(v′) = α(v) for all v ∈ V , either before or after changing α(u). If you want to remove an edge of μ(G) that’s in G, use (i); otherwise use (ii).

[See J. Mycielski, Colloquium Mathematicum 3 (1955), 161–162; H. Sachs, Einführung in die Theorie der endlichen Graphen (1970), §V.5.]

117. (a) Use the answer to (b), with each clique consisting of a single edge.

(b) Each vertex v has d options ‘v c_1j ... ckj’ for 1 ≤ j ≤ d, where the cliques containing v are {c_1,...,ck}.

(c) We save a factor of 9! = 362880 by fixing the colors of the queens in the top row. Then there are 262164 solutions, found by Algorithm X in 8.3 Tμ with method (a) but in only 0.6 Tμ with method (b).

(d) Insert ‘v′:j’ into the jth option for v, where v′ is secondary. (This reduces the running time for method (a) in part (c) to 5.0 Tμ, without fixing any colors.)

(e) Using (d) to save a factor of c!, we get (2!·1, 3!·5, 4!·520, 5!·23713820) solutions, in approximately (600, 4000, 130000, 4100000000) mems. [Monte Carlo estimates can be made for larger cases by combining exercises 86 and 122; the true branching factor at each level can be determined by rejecting options that involve illegal purification. It appears that M_6 can be 6-colored in approximately 6! · 2.0 × 10^17 ways.]

(f) Now (d) saves a factor of (c−1)!, despite having no solutions; the running times are roughly (100, 600, 5000, 300000) mems. (But then for 5-coloring M_6 it’s 45 Tμ!)

(g) There are (1! · 1, 2! · 1, 3! · (5 or 7), 4! · (1432, 1544, 1600, 2200, 2492, 2680, 3744, 4602, or 6640)) such colorings, depending on which edge is deleted.

118. In general, colorings of a hypergraph can be found with the construction of the previous exercise, but using Algorithm M and giving multiplicity [0 .. (r−1)] to each hyperedge of size r. In this case, however, there are 380 independent sets of size 16 (see exercise 7.1.4– 242); we can simply use them as options to an exact cover problem with 64 items. There are four solutions, having a curious symmetry so that only two are “essentially different”: One is shown, and the other is obtained by keeping A and C fixed but transposing the B’s and D’s.

A B C D C D A B
B D B D C A C A
C B A C D B A D
D D A B A B C C
A A D C D C B B
B C D B A C D A
C A C A B D B D
D C B A B A D C

119. Exactly three interior edges are white in every solution. Any other placement of the all-white piece defines those three edges. That leaves no way to place all three of the two-white pieces.

120. (a) Call the types 0, 1, ... , 9, and use Algorithm C to find all ways to place a given type at the center of a 5 × 5 array. There are respectively (16, 8, 19, 8, 8, 8, 10, 8, 16, 24) ways to do this; and the intersection of all solutions for a given type shows that

????????????0????2???????,0490?2023??217?8568?490??,???????0????2????????????,685680490?2032??217?8568?,21721?68569049?320??172??,320321721768568049??20???,2032??217?8568?490??032??,490490320?21721?68569049?,0320?21721?68569049?320??,1721768568049??20????2???

are the respective neighborhoods that are forced near a given type in any infinite tiling. Consequently every such tiling contains at least one 5; and if we place 5 at the origin everything in the entire plane is forced. The result is a torus in the sense of exercise 7–137, with a periodic supertile of size 12:

(b) Similarly, there’s again a unique tiling, this time with a 13-cell supertile:

121. (a) Marek Tyburec noticed in 2017 that there are no 2 × 2 solutions with βUS at lower right; similarly, there are no 3 × 4 solutions with βUS at lower left. Hence βUS can appear only in the top row, or at the left of the next-to-top row.

(b) Let (Ak, Bk, Ck, Dk) be the (2k−1)×(2k−1) tilings defined by (αa, αb, αc, αd) when k = 1, otherwise by placing (δNa, δNb, δNc, δNd) in the middle and placing Ak−1, Bk−1, Ck−1, Dk−1 at the corners as in answer 2.3.4.3–5. The unique tiling requested here has δRD in the middle and Dk−1, Ck−1, Bk−1, Ak−1 at the corners.

(c) With δRU or δLD in the middle, another solution has Ck−1, Dk−1, Ak−1, Bk−1 at the corners. With δLU or δSU, there’s a third solution with Bk−1, Ak−1, Dk−1, Ck−1 at the corners. And δSU also has 54 additional solutions with Ck−2 in the upper left corner; they use {DL, DP, DS, DT, UL, UP, UR, US, UT } in the upper half when choices need to be made, and independently {R,YR,L,P,S,T } in the lower half.

(d) Only one of each survives. As in (b), its four quadrants are D∞, C∞, B∞, A∞.

[Each of the other 86 types occurs in A_6, hence in every sufficiently large tiling. Incidentally, the “dragon sequence” (see answer 4.5.3–41) arises in the colors at the edges of A∞, B∞, C∞, D∞.]

122. A new global variable Θ, initially v, is the current “color threshold.” Every item has a new field CTH in addition to NAME, LLINK, and RLINK. That field is normally zero in primary items, although it has a special use in step C3 as described below. In secondary items, CTH will be used to undo changes to Θ.

Insert ‘CTH(i) ← Θ; if c = Θ, set Θ ← Θ + 1’ just after ‘i ← TOP(p)’ in the purify routine (55). Insert ‘Θ ← CTH(i)’ just after ‘i ← TOP(p)’ in the unpurify routine (57). Modify the commit routine (54) so that it jumps to the end of the uncommit routine (56), if COLOR(p) > Θ, without changing j or p. (The effect is to avoid committing to any option that would have set a color value greater than Θ, by jumping from step C5 into the appropriate place within step C6.*)

* Backtrack programs often run into such cases where it is permissible, even desirable, to jump into the middle of a loop. See Examples 6c and 7a in the author’s paper “Structured programming with go to statements,” Computing Surveys 6 (1974), 261–301.

Finally, change step C3 so that it never chooses an item i for which CTH(i) > Θ. That step should then go to C8 if no item is choosable. (This mechanism prohibits branching on primary items for which the assumption of total symmetry between all colors ≥ Θ isn’t yet valid. Exercise 126 has an example.)

123. When, say, m = 4 and n = 10, Algorithm C takes 49 megamems to produce 1048576 solutions. The modified algorithm (where we set v = 1) takes 2 megamems to produce 43947 solutions. (Notice that the value vectors q_1 ...qn are equivalent to the restricted growth strings a_1 ...an of 7.2.1.5–4, with qk = ak + 1.)

124. Let (x, y) denote a Δ triangle, and let (x, y)′ denote the ∇ triangle that lies immediately to its right. (Think of a square cell (x, y) that has been subdivided into right triangles by its main diagonal, then slanted and yscaled by 3/2.) For example, an m × n parallelogram has 2mn triangles (x, y) and (x, y)′ for 0 ≤ x < m and 0 ≤ y < n, Cartesianwise; the 3×2 case is illustrated.

The boundary edges of triangle (x, y) are conveniently denoted by /xy, \xy, and -xy. Then the boundary edges of (x, y) are /(x+1)y, \xy, and -x(y+1).

[A “barycentric” alternative with three coordinates is also of interest, because it’s more symmetrical: Each triangle corresponds to an ordered triple of integers (x, y, z) such that x + y + z = 1 or 2, under the correspondence (x, y) ↔ (x, y, 2 − x − y) and (x, y)′ ↔ (x, y, 1 − x − y). The twelve symmetries are then the six permutations of {x, y, z} with an optional flip between (x, y, z) and (x¯,y¯,z¯,)=(1−z,1−y,1−z).]

[One can also use “barycentric even/odd coordinates,” inspired by exercise 145, which are ordered triples (x, y, z) with |x + y + z| ≤ 1. Cases with x, y, z odd represent triangles, with (x, y) ↔ (2x−1, 2y −1, 3−2x−2y), (x, y)′ ↔ (2x−1, 2y −1, 1−2x−2y). Cases with x, y, z even represent vertices. Cases with just one even coordinate represent edges (the average of two adjacent triangles). Cases with two even coordinates could represent directed edges.]

125. Every original triangle (x, y) or (x, y)′ expands to k^2 triangles of the forms (kx + p, kx + q) or (kx + p′,kx + q′)′ for 0 ≤ p, q, p′,q′ < k. Those obtained from (x, y) have p + q < k and p′ + q′ < k − 1 (of which there are (k+12) and (k2), respectively). The others are obtained from (x, y)′.

126. Let there be 24 primary items 01’, 02, 02’, ... , 32 for the triangles, and 24 primary items aaa, aab, ... , ddd for the tiles, together with 42 secondary items \01, -02, /02, ... , /41 for the edges. There are 24 · 64 options ‘01’ aaa -02:a /11:a \01:a’, ‘01’ aab -02:a /11:a \01:b’, ‘01’ aab -02:a /11:b \01:a’, ... , ‘32 ddd -32:d /32:d \32:d’ — one for each way to place a tile. Finally, to force the boundary conditions, add another primary item ‘*’, and another option ‘* -20:a -30:a /40:a ... \10:a’.

Algorithm C finds 11,853,792 solutions, after 340 Gμ of computation; this total includes 72 different versions of every distinct solution, hence there really are just 164,636 of them (a number that was unknown until Toby Gottfried computed it in 2001).

Using exercise 119 we can remove all options for aaa except ‘20 aaa -20:a /20:a \20:a’. Algorithm C then finds 11853792/12 = 987,816 solutions, in 25 Gμ.

Furthermore, using exercise 122 (with v = b), and not allowing step C3 to branch on a tile name until Θ = e (because there’s total symmetry with respect to triangle locations but not tile names), finds every distinct solution just once, in 6.9 Gμ.

Finally, we can allow branching on aab whenever Θ ≥ c, and in general on a piece name whenever Θ exceeds all colors in its name. This reduces the runtime to 4.5 Gμ.

[MacMahon specifically designed pattern (59b) to include all three of the nonwhite solid-color triangles in the center. If we fix them in those positions, an unmodified Algorithm C quickly finds 2138 solutions. There also are 2670 solutions with those three fixed in positions {11’, 21’, 12’} instead of {12, 21, 22}.]

127. Every color appears in (3 · 24)/4 = 18 places among the triangles, hence 18 − 2k times on the border when it occurs k times in the interior of a solution. Consequently no color occurs an odd number of times on the border. That leaves 2099200 possibilities.

All of those 2099200 are actually completable. (MacMahon would have been very happy to have known this!) The number of cases can be reduced to only 4054, using the methods of Section 7.2.3, because there are 576 symmetries: cyclic shifting and/or reflection and/or permutation of colors. The Monte Carlo procedure of Algorithm 7.2.2E not only finds solutions in each of those cases, it finds oodles of them. In fact, we can be confident that every all-even-but-not-constant border specification has more than four times as many solutions as the pure-white border does.

(More precisely, the pure-white border 000000000000 has 11853792 solutions, without reducing by symmetry; the next-smallest border, 000000000011, has 48620416; the next-smallest, 000000000101, has 49941040; and so on. There are more than 100 million solutions in the vast majority of cases, but probably never more than 500 million. Incidentally, 001022021121 is the only valid color pattern that has exactly three automorphisms.)

128. We can pack them into the 11-triangle region obtained by deleting triangle (2, 1)′ from the 2 × 3 parallelogram in answer 124, in such a way that the edge colors satisfy -00 = -20, /01 = /30, -02 = -12. There are 1032 ways to do this, one of which is shown. This yields a “supertile” that nicely tiles the plane, in combination with its 180° rotation:

129. First consider rotation symmetry. Only 180° rotation applies, because of the four single-color tiles. To generate all of the strong solutions, assume that rotation changes a ↔ d, b ↔ c, and combine the options of answer 126 into pairs such as ‘02 abc -02:a /02:b \02:c 31’ bdc -32:d /41:c \31:b’. The resulting 768 options have 68,024,064 solutions (found in about 0.5 Tμ); but many of those solutions are essentially the same (that is, obtainable from each other by rotation, reflection and/or color permutation).

It’s somewhat tricky to count the essentially distinct patterns; canonical representations can be obtained by distinguishing six types of solutions: (1) 02 aaa (hence 31’ ddd) and 03 bbb (hence 30’ ccc), and /12:a or /12:c. [The cases /12:b or /12:d are equivalent to these, if we reflect and swap a ↔ b, c ↔ d.] (2) 02 aaa, 23 bbb [or equivalently 03’ bbb]. (3) 02 aaa, 13’ bbb, and \03:a or \03:c. (4) 02 aaa, and bbb in 12, 12’, 22, 22’, or 13. (5) 13 aaa, 02’ bbb, and \12:a or \12:c. (6) 13 aaa, 12 bbb. Each type is easy, yielding 80768+164964+77660+819832+88772+185172 = 1417168 solutions.

[Notice that the illustrated example of strong symmetry actually tiles the plane without rotation; that is, it has -04 = -20, -14 = -30, /03 = /41, ... , \10 = \32. Exactly 40208 of the essentially distinct solutions satisfy this additional proviso.]

To generate the weak solutions, introduce new secondary items bxy, bxy′ for each triangle (x, y) or (x, y)′ with y > 1, representing color changes within the triangle. Typical options are now ‘02 aad -02:a /02:a \02:d b02:5’, ‘02 aad -02:a /02:d \02:a b02:3’, ‘02 aad -02:d /02:a \02:a b02:6’, ‘02 abc -02:a /02:b \02:c b02:7’, ‘31’ bdc -32:c /41:b \31:d b02:7’, ‘31’ ccd -32:c /41:c \31:d b02:5’. We may assume that ddd is opposite aaa, ccc is opposite bbb. Algorithm C generates each weak-not-strong solution twice, each strong solution once; the six types yield a total of 24516 + 45818 + 22202 + 341301 + 44690 + 130676 = 609203 weak-not-strong solutions.

Turning now to reflections of the hexagon, there are two essentially different possibilities: Top-bottom reflection preserves the values of four edges, but all triangles change; left-right reflection preserves the values of four triangles and two edges. Therefore strong reflection symmetry is impossible. (In the first case, all triangles change, hence all colors change. In the second case, two colors must be fixed. With colors a and d fixed but b ↔ c, eight triangles aaa, aad, abc, acb, bcd, bdc, dda, ddd must be fixed.)

Weak symmetry under top-bottom reflection can be assumed as before to take aaa to ddd, bbb to ccc. Again there are six types: [1] 02’ aaa, 22’ bbb, -13:a or -13:c. [2] 02’ aaa, bbb in 12’, 03’, 13, 13’, or 23. [3] 12’ aaa, bbb in 03 or 03’. [4] 03 aaa, 23 bbb, -13:a or -13:c. [5] 03 aaa, bbb in 13 or 13’. [6] 03’ aaa, 13’ bbb, -13:a or -13:c. Surprisingly, some placements are “special”: They have strong rotational symmetry, as well as weak top-down symmetry! Algorithm C, which generates the special ones once and the others twice, produces respectively (88, 0, 0, 98, 0, 75) + 2(1108, 12827, 8086, 3253, 12145, 4189) solutions. Here are examples of the 88 + 98 + 75 = 261 special placements, which belong simultaneously to types [1] and (5), [4] and (3), [6] and (1):

Weak left-right symmetry is similar, but there now are some fixed triangles. If aaa is fixed, assume that ddd is also fixed; three such types arise, with 46975 + 35375 + 25261 = 107611 solutions. Otherwise assume that ddd is opposite aaa; six types of this kind yield (75, 0, 98, 0, 0, 88) strong and (3711, 56706, 5889, 60297, 38311, 9093) non-strong solutions. So there’s a grand total of 281618 essentially distinct weak-not-strong placements with left-right symmetry — of which 194 are top-down symmetric too.

[Arrangements that have strong and weak symmetry were first discovered by Kate Jones, who presented them in the 1991 user manual for Multimatch® III, an attractively produced set of triangular tiles.]

130. The nicest coordinate system for an octahedron is probably to number the faces 000, 001, ... , 111 in binary, and to let the vertices be {0**, 1**, *0*, *1*, **0, **1}; the edges are {xy*,x*y, *xy} for x, y ∈ {0, 1}. Construct 512 options ‘000 aaa *00:a 0*0:a 00*:a’, ‘000 aab *00:a 0*0:b 00*:a’, ‘000 aab *00:b 0*0:a 00*:a’, ... , with face-name items 000, ... , 111 primary and tile-name items aaa, ... , ddd secondary. Algorithm C quickly finds 2723472 solutions, which include 45356 distinct sets of eight. Those 45356 sets become, in turn, new options for Algorithm X (or C), with 24 primary tile-name items; now we get 1615452 solutions, which are the desired partitions.

Many symmetries are present, of course; we’ll study how to distinguish nonisomorphic representatives in Section 7.2.3. One of the most interesting solutions,

has four color-swap symmetries, with all the solid-color triangles on one octahedron.

131. (a) Each triangle edge is either a (straight) or b (a wave) or c (a hump) or d (a dip). We can set this up with options and items as in answer 126, except that the edge-match condition is now a ↦ a, b ↦ b, c ↦ d, d ↦ c; to get proper matching, the options of ∇ triangles should state the mate color, as in ‘01’ abc -02:a /11:b \01:d’.

Every solution corresponds to 24 equivalent solutions, because we get a factor of 6 by rotating the hexagon, a factor of two by interchanging humps with dips, and another factor of two by reflection. (Reflection is a bit tricky, because a wave becomes an anti-wave when a piece is flipped over. However, every reflected piece has its own anti-piece, which yields the desired anti-solution.) Thus we can force aaa to be in position 02. Treating c and d symmetrically as in answer 126 (with v = c) produces exactly 2,231,724 canonical solutions and needs only 30 gigamems of running time.

[This puzzle is manufactured by Kadon Enterprises under the name Trifolia®.]

(b) A similar setup, letting c and d represent 0 spots and 3 spots so that it’s easy to treat them symmetrically, now has mates a ↦ b, b ↦ a, c ↦ d, d ↦ c; hence one option is ‘01’ abc -02:b /11:a \01:d’. The boundary colors in directions / and \ are a; in direction - they are b. The solutions to this problem typically form groups of eight (not 24): We can swap c ↔ d, reflect left-right, reflect top-down, or rotate by 180°; the latter two are combined with swapping a ↔ b. Without attempting to remove any symmetries, we get 3,419,736,176 solutions, after 20.6 teramems of computation.

Left-right reflection always gives a distinct solution, whether we swap c ↔ d or not (because there are at least eight pieces that stay fixed, and only four places to put them). But the illustrated example shows that some solutions are fixed under 180° rotation; we can find them by adding 15 new primary items, such as #/23, and 15 · 4 new options, such as ‘#/23 /23:x /20:x’ for x ∈ {a, b, c, d}. Altogether 18656 solutions have that symmetry; such cases form groups of four, not eight. Similarly, 169368 cases turn out to have top-down symmetry. It follows from “Burnside’s lemma” that the total number of essentially different solutions is (3419736176 + 18656 + 169368)/8 = 427490525.

To double the speed of all these computations, take v = c in exercise 122.

132. This challenging problem was first resolved by Peter Esser in April 2002, and presented online at www.polyforms.eu/coloredpolygons/triindex.xhtml#trios24. [See JRM 9 (1977), 209. One can show that the only solutions to the Diophantine equation d + d(d−1) + d(d−1)(d−2)/3 = m^2 are d = 1, 2, and 24, using advanced methods found in N. P. Smart, The Algorithmic Resolution of Diophantine Equations (1998).]

133. This problem is like exercise 126, but considerably simpler because squares are easier than triangles. There are 24 · 81 options ‘00 aaaa h00:a v10:a h01:a v00:a’, ... , ‘53 ccba h53:a v63:c h54:c v53:b’, where hxy and vxy denote the horizontal and vertical edges between squares. We save a factor of 4 by limiting aaaa to four positions on the border, and another factor of 2 by making b and c equivalent (exercise 122 with v = b). The resulting 13328 solutions are found in 15 Gμ.

[Today it’s easy to count them; but this problem has a tortured history! T. H. O’Beirne missed two of the 20 possible ways to place the internal white edges, when he analyzed the situation by hand in New Scientist 9 (2 February 1961), 288–289. A few years later, the problem of solution counts for MacMahon squares was probably the very first large computation ever undertaken at Stanford Artificial Intelligence Laboratory; Gary Feldman found 12261 placements, during a 40-hour computer run (see Stanford AI Project, Memo 12 (16 January 1964), 8 pages). That number was believed to be correct until May 1977, when the true value was obtained by H. Fernández Long in Argentina.]

Instead of denoting squares by xy and edges by {hxy, vxy}, it’s convenient to use “even/odd coordinates” instead (see exercise 145). In that system, a pair of odd numbers (2x+1)(2y+1) denotes a square, and the edge between two adjacent squares is represented by the midpoint between them. For example, the 24·81 options sketched above would then take the form

‘11 aaaa 01:a 12:a 21:a 10:a’, ... , ‘b7 ccba a7:a b8:c c7:c b6:b’.

Such coordinates are easier to work with under reflection and rotation.

134. (O, P, Q, ... , Z) occur respectively (0, 1672, 22, 729, 402, 61, 36, 48, 174, 259, 242, 0) times, sometimes twice in the same solution; one solution features four pentominoes.

[Kate Jones introduced such questions in the Multimatch® I user manual (1991).]

135. Indeed, the total number of solutions is enormous; Monte Carlo estimates predict ≈ 9 × 10^8 of them for any fixed placements of aaaa, bbbb, cccc that aren’t obviously impossible. Therefore it’s natural to impose extra conditions. The elegant wrapping below permutes colors cyclically and has solid colors on every edge of the cube! Investigations by H. L. Nelson, F. Fink, and M. Risue˜no showed that 61 such solutions are possible; see W. E. Philpott, JRM 7 (1974), 266–275. See answer 145 for an even/odd coordinate system that is useful for representing this problem internally.

(Wrapping the surface of a symmetrical polyhedron is a nice way to avoid awkward boundary conditions when arranging MacMahon-like tiles. Dario Uri devised 39 such problems in 1993, together with ingenious mechanical frames for building the results. Here, for example, are a rhombic triacontahedron (30 rhombuses) and a stellated dodecahedron (60 isosceles triangles), based on all possible ways to put distinct colors from {red, green, blue, yellow, black} on the edges. His report “Tessere di Mac Mahon su superfici tridimensionali” is online at www.uriland.it.)

136. The main challenge is to find a good way to represent the faces and edges of a dodecahedron. Perhaps the nicest is to represent the faces by vertices of an icosahedron, with the three-dimensional coordinates (0, (−1)bϕ, (−1)c)σa, where (x, y, z)σ = (z, x, y); let abc stand for this face, for 0 ≤ a < 3 and 0 ≤ b, c < 2. A face is adjacent to its five nearest neighbors; we can represent the edge between abc and a′b′c′ as the midpoint (abc + a′b′c′)/2. These 30 midpoints have two forms, either ab = (0, (−1)bϕ, 0)σa or abcd=12((−1)b,(−1)cϕ,(−1)dϕ2)σa. The corresponding XCC problem can now be formulated as usual, with 120 options for each face. For example, a typical option for face 201 is ‘201 01243 20:3 1100:0 2001:1 2101:2 1110:4’.

We can force the first tile to be in a particular place by default. Algorithm C needs only 9 megamems to solve the resulting problem, and produces 60 solutions.

Of course many of those solutions are equivalent. There are 120 transformations that preserve the dodecahedron and icosahedron as represented above, generated by three reflection matrices and two orthogonal matrices,

D0=(−1000+1000+1),D1=(+1000−1000+1),D2=(+1000+1000−1),P=(001100010),Q=12(1−ϕ1/ϕϕ1/ϕ−11/ϕ1ϕ)

Applying any combination of these, and remapping the colors to agree with the default placement, gives an equivalent solution. It turns out, as Conway discovered by hand(!), that there are just three inequivalent solutions, having respectively 4, 6, and 12 automorphisms (hence occurring 30, 20, and 10 times in the output of Algorithm C):

[See M. R. Boothroyd and J. H. Conway, Eureka: The Archimedeans’ Journal 22 (1959), 15–17, 22–23. Conway has named them the “quintominal dodecahedra.”]

137. (a) This is an easy application of Algorithm C, with 14 + 12 items and 7 · (1 + 6 × 6) = 259 options. (Clever reasoning also allows it to be established by hand, with a search tree of size 15.)

(b) No. Again Algorithm C gives the answer quickly.

(c) Thousands of random trials indicate that about 93% of the (1207) choices have no solution; about 5% have just one solution; about 1% have two solutions; and the remaining 1% have three or more.

(d) About 0.4% of all cases work, as in the example shown.

Historical notes: Milton Bradley Company introduced Drive Ya Nuts in 1970; the name of its inventor has unfortunately been forgotten. It was preceded by a much more difficult puzzle with 19 hexagons in three concentric rings, called Super Dom [H. Hydes, British Patent 149473 (19 August 1920)], and by several similar puzzles [H. Hydes and F. R. B. Whitehouse, British Patent 173588 (29 December 1921); G. H. Haswell, U.S. Patent 1558165 (20 October 1925)], featuring both kinds of edge-matching rules.

138. (a) We can name the tiles ABcd, ABdc, ACbd, ... , DCba. Assuming that ABcd is in the top left corner, a straightforward application of Algorithm C (with 2118 options involving 48 + 48 items) will output 42680 solutions, in 13 gigamems. As in other such problems, however, these outputs include many that are essentially the same. Up to 96 equivalent solutions are related by the operations of shifting any cell to the top-left position and/or flipping horizontally and/or flipping vertically, then remapping the colors. For instance, the given example has six automorphisms: We can shift it two columns right, then map A ↦ C ↦ D ↦ A, a ↦ c ↦ d ↦ a; we can also shift two rows down, reflect left-right, then A ↔ D and a ↔ d. Hence it contributes 96/6 = 16 cases to the total of 42680. Altogether there are (79, 531, 5, 351, 6, 68, 12, 4) cases with respectively (1, 2, 3, 4, 6, 8, 12, 24) automorphisms, hence 79 + 531+5+351+6+68+ 12+4 = 1056 essentially different solutions. One with 24 symmetries is shown below (it leads to itself if we move right 1 and down 2, and/or reflect horizontally or vertically).

(b) Now Algorithm C, given 1089 options involving 49+60 items, quickly finds just six solutions — three different pairs related by transposition, each of which is symmetric under 90° rotation, all with heads and tails in the same places.

(c) Take any of the three solutions to (b), reflect it top-down, interchange heads with tails, and swap B ↔ D, b ↔ d. For example, the dual of the given solution is shown below. Alternating all-heads with all-tails, in checkerboard fashion, yields uncountably many tilings of the plane.

[These tiles are believed to have originated in 1990 with a puzzle called “Super Heads & Tails,” designed by Howard Swift and produced in a limited edition.]

139. (a) Say that two sets of nine are essentially the same if one can be obtained from the other by remapping the colors, and/or reflecting all of the pieces, and/or interchanging heads with tails. For example, 4! × 2 × 2 = 96 different choices of nine are equivalent to the set

By considering canonical forms, as in exercise 138(a), we find 14124 equivalence classes, of which (13157, 882, 7, 78) have the respective sizes (96/1, 96/2, 96/3, 96/4).

(b) There are exactly (9666, 1883, 1051, 537, 380, 213, 147, 68, 60, 27, 29, 9, 24, 4, 8, 2, 5, 4, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1) classes with (0, 1, 2, ... , 27) solutions; the amazing one with 27 is represented by (∗) above. Two of the 1883 puzzles with unique solutions are particularly interesting because they have four automorphisms:

In each case we can flip the pieces and/or swap heads ↔ tails, then remap the colors to get the original tiles.

[This problem was first solved by Jacques Haubrich in 1996, who considered color remapping only (hence he had 54498 equivalence classes). Haubrich has collected 435 inequivalent puzzles, from around the world, that consist of nine tiles with two heads opposite two tails. But only 17 of them have all tiles different and all four objects different on each tile; for example, at least one tile such as ABcb is usually present. The first “pure” HHtt puzzle in his collection was made by the Hoek Loos company in 1974.]

140. (a) We save a factor of 4! by applying exercise 122 with v = a. Then Algorithm C gives respectively (10, 5, 6) solutions. The true numbers, however, are (5, 3, 3), because the shapes are symmetrical — and because the middle solution has an additional symmetry: It goes into itself if we rotate by 180° and permute the colors.

(b) The scaled-up versions of are impossible. But we have

with respectively (4, 4, 3) solutions; and there are unique solutions to the other five:

(c) These shapes, with respectively (7, 9, 48, 2, 23, 28, 18) solutions, are a bit easier to handle. The “wave” has six solutions with central symmetry; the “bar” has four.

[Vertex-colored triangles have been named ‘Trioker’ by Marc Odier; see French Patent 1582023 (1968), U.S. Patent 3608906 (1971), and the book Surprenant Triangles, which he published with Yves Roussel in 1976. They also are sold as Multimatch® IV.]

141. (a) Using exercise 122 with v = a yields respectively (138248, 49336, 147708) solutions in (1390, 330, 720) gigamems. Then we divide by (8, 4, 4) to remove symmetries of the board, getting (17281, 12334, 36927) solutions that are essentially distinct. [These numbers were first computed by Toby Gottfried in (1998, 1999, 2002). He had been interested in the puzzle ever since seeing the 5 × 5 version that was sold by Skor-Mor in 1970 under the name “Nitty Gritty.” The puzzle is extremely difficult to solve by hand, in spite of the many solutions; Langford himself was unable to solve the 3 × 8 case.]

The 12334 solutions for 4 × 6 include 180 that have matching colors at the left and right. Each of these patterns therefore tiles a “cylinder”; and the 180 form 30 families of 6 that are equivalent to each other by rotating the cylinder. Similarly, 1536 of the 36927 solutions for 3 × 8 are cylindrical, making 192 families of 8. The example illustrated is one of 42 that have the same solid color at both left and right.

(b) Any solution can be used to tile the plane in combination with its mirror reflections and its 180° rotation (which is a reflection of a reflection).

The 17281 solutions include 209 for which the hole is surrounded by a single color. Six of these have matching colors at two opposite sides; the one illustrated will tile the plane in conjunction with its mate, which is obtained by swapping b ↔ c.

The 4 × 6 example illustrated is the unique solution for which both pairs of opposite sides induce exactly the same color partition (the restricted growth strings 0121120 and 01220). Thus it too will tile the plane together with its b ↔ c mate.

[Vertex-matched squares, with incomplete sets of tiles, first appeared in puzzles devised by E. L. Thurston, U.S. Patents 487797 (1892), 490689 (1893).]

142. Each boundary between the square cells containing octagons now has two secondary items that receive color. For example, a typical option for Algorithm C is now ‘10 aabc a_10:a r_10:a l_11:b a_11:b b_21:c l_21:c r_20:a b_20:a’, where axy, bxy, lxy, and rxy denote the half edges above, below, left, and right of (x, y). The number of solutions, again using exercise 122 with v = a, is 2 · (132046861, 1658603, 119599) in cases (i), (ii), (iii), found in (2607, 10223, 77) gigamems. Case (i) includes 2 · (193920, 10512, 96) “cylindrical” arrangements in which the colors match at top/bottom, left/right, both; one of the 96 “toroidal” examples is shown. Case (ii) includes 2 · 5980 cylindrical arrangements that match at left/right. Case (iii) has no cylindrical examples.

[Many other possibilities arise, because neighboring octagons can match without lying in a square grid. Kadon Enterprises offers attractive sets called ‘Doris®’.]

143. (a) simplex (8, 6, 8, 2, 0, 0, 0); simplex (7, 4, 7, 3, 0, 0, 0); simplex (5, 5, 5, 4, 0, 0, 0).

(b, c) Nonnegative integers x_0x_1x_2x_3x_4x_5 define such a polygon if and only if the boundary path returns to its starting point, which means that x_0 + x_1 = x_3 + x_4 and x_1 + x_2 = x_4 + x_5. Rotating by 60° replaces x_0x_1x_2x_3x_4x_5 by x_5x_0x_1x_2x_3x_4; reflecting left ↔ right replaces x_0x_1x_2x_3x_4x_5 by x_0x_5x_4x_3x_2x_1. Hence we get a canonical form by insisting that x_0 ≥ x_3 ≥ x_5 ≥ x_1: Every sequence of nonnegative integers (a, b, c, d) with a ≥ b ≥ c ≥ d defines the boundary x_0x_1x_2x_3x_4x_5 of a unique convex triangular polygon, where x_0 = a, x_1 = d, x_2 = a − b + c, x_3 = b, x_4 = a − b + d, x_5 = c. Furthermore, that polygon contains exactly N = (a + c + d)2 − b^2 − c^2 − d^2 triangles.

Given N, the following algorithm visits all relevant (a, b, c, d). For c = 0, 1, ... , while 2c^2 ≤ N do the following: For d = 0, 1, ... , while d ≤ c and 2c(c +2d) ≤ N, let x = N +c^2 +d^2. If x mod 4 ≠ 2, for every divisor q of x such that q ≡ x (modulo 2) and q^2 ≤ x, set a ← (x/q+q)/2−c−d and b ← (x/q−q)/2. Visit (a, b, c, d)if a ≥ b and b ≥ c.

When N = 24 this algorithm visits six (a, b, c, d), namely (7, 5, 0, 0), (5, 1, 0, 0), (12, 12, 1, 0), (6, 6, 2, 0), (2, 2, 2, 2), (4, 4, 3, 0). The fourth, sixth, and second are the shapes of exercise 140. The other three cannot be tiled properly with Langford’s 24 tiles.

[See OEIS sequence A096004, contributed by P. Boddington in 2004.]

(d) Yes. One way is simplex (a + c + d, a + c, a + d, a − b + c + d, 0, 0, 0).

144. The constraints are severe, because a solid color is needed at transitions between regimes. Algorithm C (with v = a as in answer 142) quickly finds 2·102 solutions to (ii). But surprisingly many arrangements arise in case (i); Algorithm C finds 2 · 37586004 of them, not so quickly (643 teramems)!

(These tiles suggest many intriguing questions. For example, suppose we restrict consideration to making a big hexagon from 24 small ones. There are 2^24 ways to specify whether each position should be matched at vertices or edges; but very few of those specifications are actually realizable. Can the realizable ones be nicely characterized?)

145. Suppose 0 ≤ i ≤ l, 0 ≤ j ≤ m, and 0 ≤ k ≤ n. Let (2i, 2j, 2k) represent vertex (i, j, k); let (u + v)/2 represent the edge between adjacent vertices u and v; let (a + b)/2 represent the face containing parallel edges a and b; let (e + f)/2 represent the cell containing parallel faces e and f. Thus, the triple (x, y, z) represents a vertex, edge, face, or cell when it has respectively 0, 1, 2, or 3 odd coordinates.

For example, (2i, 2j+1, 2k) represents the edge between vertices (i, j, k) and (i, j+1, k); (2i+1, 2j, 2k+1) represents the face whose vertices are (i, j, k), (i+1, j,k), (i, j, k+1), (i+1, j,k+1); and (2i+1, 2j+1, 2k+1) represents the cell whose eight vertices are (i +(0 or 1), j +(0 or 1), k +(0 or 1)).

Notice that (a + b)/2 represents the vertex between adjacent parallel edges a and b; (e + f)/2 represents the edge between adjacent parallel faces e and f; (p + q)/2 represents the face between adjacent cells p and q.

(We can use a similar convention in two dimensions, as an alternative to the ‘H’ and ‘V’ items in situations like answer 109.)

146. (a) Each color occurs four times on the “visible” faces and at most twice on the “hidden” faces. So the five adjacencies account for all six occurrences of five colors.

(b) For every partition of {a, b, c, d, e, f} into three pairs {u, u′}, {v, v′}, {w, w′}, there are two chiral cubes having u opposite u′, v opposite v′, w opposite w′. Order the colors so that u < u′, u < v, v < v′, v < w, v < w′; there are 30 ways to do this. The cube named uu′vv′ww′ is the one that can be placed with u on top, u′ on the bottom, v in front, v′ in the back, w at left, w′ at the right. For example, the cubes in (∗) are named aebfcd, acbfde, acbdef, afbdec, abcedf, aebcfd.

(c) We can set this up for Algorithm C by specifying 6 · 30 · 24 options, one for each cube position, cube name, and cube placement. There are 6 primary items for the positions; 30 secondary items for the names; 4 · 6 primary items uc, dc, fc, bc for colors on the top, bottom, front, and back, where c ∈ {a, b, c, d, e, f}; and 6 secondary items hk for the colors hidden between positions k and k + 1. For example, the leftmost cube in (∗) corresponds to the option ‘1 aebfcd ua de fb bf h_0:c h_1:d’.

If we eliminate all but one option for position 1 (thus saving a factor of 720), there are 2176 solutions. Each solution is, however, potentially equivalent to 95 others, because there are 16 possible rotations/reflections together with 6 cyclic permutations (followed by remapping the colors of the leftmost cube). For example, the solution illustrated has 12 such automorphisms. Further study shows that only 33 solutions are “essentially different” — of which (17, 9, 3, 1, 3) have (1, 2, 4, 6, 12) automorphisms.

(d) Yes, in lots and lots of ways. The 720 · 2176 solutions obtained without fixing the leftmost cube involve 15500 different 6-tuples of cubes; and the exact cover problem for which those 6-tuples are the options has 163,088,368 solutions.

[This problem was posed by Martin Gardner in Scientific American 204, 3 (March 1961), 168–174 (long before the “Instant Insanity” craze), and he extended it to question (c) in Scientific American 235, 3 (September 1978), 26. A solution to (d) that involves five symmetrical arrangements was found by Zoltan Perjés in 1981; see Gardner’s book Fractal Music, Hypercards, and More (1992), 97.]

147. (a) The “even/odd coordinates” of exercise 145 are ideal for representing the cube positions and the faces between them. For example, the colors in the 1 × 2 × 2 brick that was illustrated with the exercise are nicely represented by the 3 × 5 × 5 array

[.......a.a........a.a.......][.d.d.c.e.c.f.f.c.d.c.e.e.][......b.b.......b.b.......],
where entry (0, 1, 1) = a, entry (1, 0, 1) = d, entry (1, 1, 0) = c, ... , entry (2, 3, 3) = b. The cubes in positions (1, 1, 1), (1, 1, 3), (1, 3, 1), (1, 3, 3) of this example have the respective names abcedf, abcefd, abcdfe, abcdef. In a similar way an l × m × n brick has colors represented by a (2l +1) × (2m +1) × (2n + 1) tensor; and the tensor

[............a.a.a.a.a.............a.a.a.a.a.............a.a.a.a.a............][.c.c.c.c.c.b.f.d.f.d.b.e.e.b.b.f.b.c.d.f.e.b.f.f.e.d.d.b.e.d.b.e.b.c.c.c.c.c.][............d.b.e.e.e.............d.b.c.c.c.............d.b.f.f.f............][.c.c.c.c.c.b.f.d.b.f.b.e.e.f.d.d.b.c.d.e.f.b.f.f.b.b.e.b.e.d.e.d.b.c.c.c.c.c.][............a.a.a.a.a.............a.a.a.a.a.............a.a.a.a.a............]

represents a “magnificent brick” whose faces are colored a, b, c (each twice).

(b) Let there be lmn primary items (2i + 1)(2j + 1)(2k + 1) for the cube positions, 30 secondary items for the cube names, and lm(n+1) + l(m+1)n +(l+1)mn secondary items xyz for the cube faces, where 0 ≤ x ≤ 2l, 0 ≤ y ≤ 2m, 0 ≤ z ≤ 2n, (x mod 2) + (y mod 2) + (z mod 2) = 2. For example, the option for position 135 in solution (a) is ‘135 acbefd 035:a 125:b 134:d 136:f 145:e 235:c’. We also introduce six primary items to enforce the rule about solid colors on the brick’s faces. Each of them has six options, one for each color c; for example, the options for the top face are ‘top 101:c 103:c 105:c 107:c 109:c 301:c 303:c 305:c 307:c 309:c’. The number of solutions is reduced by a factor of 720 if we remove all but one of the 720 options for position 111.

It turns out that the brick’s face colors have an interesting property in every solution: A repeated face color occurs only on opposite, parallel faces. The example 1×2×2 brick has face colors ab×cc×de; the 2×3×5 brick in (a) has colors aa×bb×cc.

A brick is considered to be essentially the same as any other that’s obtained from it by rotation, reflection, and/or permutation of colors. The example 1 × 2 × 2 brick above has 8 automorphisms; for example, we can reflect top ↔ bottom and swap d ↔ e. The 2 × 3 × 5 brick above has 2 automorphisms: The nontrivial one reflects front ↔ back, top ↔ bottom, e ↔ f.

There’s another 1 × 2 × 2 brick, whose face colors are ab × cd × ef. It has 16 auto-morphisms. Thus it occurs only once among the three solutions found by Algorithm C when (l, m, n) = (1, 2, 2); the other two solutions are equivalent to each other.

There’s a unique 1 × 2 × 3 brick, easily found by hand. It has colors ab × cc × dd, and 8 automorphisms. (Clearly 1 × m × n is possible only if mn ≤ 6.)

The 2 × 2 × 2 bricks are especially interesting because MacMahon himself and his friend J. R. J. Jocelyn considered this case (with six different face colors), when they introduced the 30 6-color cubes in U.K. Patent 8275 of 1892. They observed that one can choose any “prototype” cube and replicate it at twice the size, by assembling eight of the other cubes. This can be done in two ways — using, in fact, the same eight cubes. But those two solutions are isomorphic, in 24 different ways. [See Proc. London Math. Soc. 24 (1893), 145–155. Their 8-cube puzzle was sold under the name “Mayblox.”]

Gerhard Kowalewski, in Alte und neue mathematische Spiele (1930), 14–19, found a 2 × 2 × 2 brick with face colors aa × bb × cd. Ferdinand Winter, in Mac Mahons Problem: Das Spiel der 30 bunten Würfel (1933), 67–87, found another, with face colors aa × bc × de. And there’s also a fourth solution, having Winter’s face colors:

MacMahonKowalewski[......a.a.......a.a......][.c.c.e.b.f.f.e.e.b.f.d.d.][......d.d.......c.c......][.c.c.e.a.f.f.e.e.a.f.d.d.][......b.b.......b.b......];[......a.a.......a.a......][.b.b.c.e.d.f.f.c.e.d.b.b.][......d.c.......d.c......][.b.b.c.f.d.e.e.c.f.d.b.b.][......a.a.......a.a......];WinterFourth[......a.a.......a.a......][.b.b.d.c.e.e.d.d.f.e.c.c.][......f.f.......b.b......][.b.b.d.c.e.e.d.d.f.e.c.c.][......a.a.......a.a......];[......a.a.......a.a......][.b.b.d.f.e.e.d.d.f.e.c.c.][......c.c.....b.b......][.b.b.d.f.e.e.d.d.f.e.c.c.][......a.a.......a.a......].

These solutions have respectively (24, 8, 4, 8) automorphisms; hence Algorithm C finds 48/24 + 48/8+48/4+48/8 = 26 solutions to the case l = m = n = 2.

Larger cases have solutions that are, perhaps, even more remarkable; but there’s room here for only a brief summary. For each feasible case of l×m×n bricks with particular face colors, we list the number of different solutions with (1, 2, 4, 8) automorphisms. Case 2 × 2 × 3: aa × bb × cc, (0, 0, 1, 0); aa × bc × dd, (0, 2, 6, 1); aa × bc × de, (0, 1, 6, 0); ab × cd × ee, (0, 1, 2, 0); ab × cd × ef, (0, 0, 2, 0). Case 2 × 2 × 4: aa × bb × cc, (0, 0, 1, 0); aa × bb × cd, (0, 0, 1, 0); aa × bc × dd, (0, 3, 4, 2); aa × bc × de, (0, 11, 14, 2); ab × cd × ee, (0, 2, 2, 3); ab × cd × ef, (0, 1, 1, 1). Case 2 × 2 × 5: aa × bc × dd, (0, 5, 4, 0); aa × bc × de, (0, 18, 9, 0); ab×cd×ee, (0, 0, 1, 0); ab×cd×ef, (0, 2, 5, 1). Case 2×3×3: aa×bb×cc, (2, 15, 4, 0); aa×bb×cd, (4, 8, 1, 0); aa×bc×de, (1, 4, 1, 2). Case 2×3×4: aa×bb×cd, (6, 8, 1, 0); aa × bc × de, (0, 6, 0, 0); ab × cc × dd, (0, 4, 2, 0); ab × cc × de, (0, 2, 0, 0); ab × cd × ee, (0, 2, 0, 0); ab × cd × ef, (0, 7, 0, 0). Case 2 × 3 × 5: aa × bb × cc, (0, 2, 0, 0).

(Conspicuous by its absence is the case l = m = n = 3. There’s no 3 × 3 × 3 brick, although we can come close: A 3 × 3 × 3 without a corner can be made from 26 of the 30; or without the middle cube and the one above it, from 25.)

148. There are eleven such cubes, and they can be matched in many pleasant ways:

[........................c.....................][...............a.....a.c.....c...............][.......a.b.a........b.a.b........a.b.a.......][.a.a.a.a.b.a.c.c.b.c.a.b.a.c.b.a.a.a.c.b.c.c.c.c.][.......b.c.b........c.b.c........b.c.b.......][...............b.....b.c.....c...............][.....................c.....................]

149. Label the vertices with nonnegative barycentric coordinates wxyz, where w + x + y + z = 3. Also label the ten unit tetrahedra with barycentric coordinates stuv, where s + t + u + v = 2; the vertices wxyz of tetrahedron stuv are then stuv + {1000, 0100, 0010, 0001}. Introduce ten primary items stuv for the tetrahedra, and ten more abcd, abdc, abce, adec, ... , bcde, bced for the different colorings. And introduce 20 secondary items wxyz for the vertices.

Then the admissible vertex colors are the solutions to the XCC problem with 1200 options ‘stuv α v_1:p_1 ... v_4:p_4’, where α is a coloring, v_1v_2v_3v_4 are the vertices of stuv, and p_1p_2p_3p_4 is an even permutation of α’s colors. Curiously, this problem has 2880 solutions (found in 500 Mμ) — and they’re all equivalent to the one below, under the 5! 4! = 2880 automorphisms present.

(This problem was posed in 2015 by J. McComb, and solved by J. Scherphuis.)

150. Notice that there are fourteen distinct pieces, with four pairs of two. So we use Algorithm M, with 14 primary items for pieces and 64 for cells. We also introduce secondary items for edges between cells, with colors to indicate the presence or absence of links. The final two pieces must obviously be adjacent, hence we can combine them into a “super-piece” of size 11; then all interfaces between adjacent cells are identical. We can remove symmetry by forcing the super-piece to be in one of 18 positions.

Then 43 solutions are found, in 7 Gμ. Here are some typical examples:

Solution (i) appears in Hoffmann’s Puzzles Old and New, puzzle 3–18. Solution (iii) avoids most of the lower left quadrant, and solution (iv) avoids the entire right column. If we ignore blank spaces, the links form eight different paths, all of length 34. Paths (i), (ii), (iii), (iv) occur in respectively 1, 15, 9, 3 of the 43 solutions. [The Endless Chain Puzzle was distributed circa 1887 by Reason Manufacturing Company.]

151. (a) The key idea is to start by factoring this problem, by considering only the task of edge-matching between adjacent dominoes, while ignoring the loop details.

Algorithm M applies, with primary items 1–9 and a–i for the distinct on-off patterns of attachment points, as well as primary items ij for each cell to be covered (0 ≤ i < 8, 0 ≤ j < 9), and two special primary items H, V. There are 63+64 secondary items hij and vij, to indicate path/nopath at internal attachment points. Typical options:

‘a 10 11 v_12:1 h_21:1 h_20:1, h_10:0 h_11:1 H’,
‘a 11 21 h_11:0 v_12:0 v_22:1, h_31:1 v_21:1 v_11:1 V’;
the goal is to find an exact cover with multiplicities 1 for patterns 1–9, multiplicities 3 for patterns a–i, and multiplicities 18 for H and V. (There are millions of solutions.)

Once that task is solved, we need to assign the actual dominoes whose subpaths jointly define a single loop. A (nontrivial) program, whose structure has a lot in common with Algorithm X, will find such assignments in microseconds (although a full day might be needed to actually write that program).

(b) Now H and V should have multiplicities 32 and 4. (Also, we can save about half of Algorithm M’s running time by omitting vertical placements at odd height.) The algorithm finds 6420 solutions; suitable domino assignments are then found in a flash.

[These 36 path dominoes were first studied by Ed Pegg Jr. in 1999, and first placed into a single-loop 8 × 9 array by Roger Phillips later that year.]

152. This (factored) problem is like the previous one, but with an additional pattern j of multiplicity 11, and a blank pattern of multiplicity 1, but without H or V. One needs to be lucky to find a solution; the author struck it rich with Algorithm M after 35.1 Tμ.

[Notice that exactly 32 of the 48 path dominoes have no crossings. Thus it is irresistible to try to place them on a chessboard, so as to form a single noncrossing loop. Unfortunately, Algorithm M tells us that such a mission is impossible, even with multiple loops, because the corresponding factored problem has no solution. Something interesting, however, can surely be done with those 32.]

153. (a) Algorithm M quickly verifies the uniqueness of the solution below, if we add a blank monomino of multiplicity 4. [“Line puzzles” like this were invented by Bill Darrah; several of his ingenious designs were made by Binary Arts in 1994 and 1999.]

(b) There are 30 patterns, three for each distinct choice of three connection points.

(c) Trials with random choices of respectively (2, 2, 4) sets of (2, 3, 4) distinct connection points usually give no solutions at all. But one of the author’s first 1000 trials was suitable, and it led to a nice puzzle whose solution is shown.

154. The integer solutions to P (n) = n(n +1)2(n +2)/12 = m^2 involve perfect squares u^2 and v^2 with v^2 ≈ 3u^2. If |v^2 − 3u^2| is sufficiently small, v/u must be a convergent to the continued fraction 3=1+//1,2,1,2,1,2,1,2,...// (see exercise 4.5.3–42).

Pursuing this idea, let θ=2+3, θ^=2−3, 〈an〉=〈(θn+θ^n/2)〉=〈1,2,7,26,97,...〉 and 〈bn〉=〈(θn−θ^n)/(23)〉=〈0,1,4,15,56,...〉. Notice that an2=3bn2+1; (an + 3bn)2 = 3(an + bn)2 − 2. We find that P (n) is a perfect square if and only if n=6bm2 for some m (thus n = 0, 6, 96, 1350, 18816, ... ) or n = (am + 3bm)2 for some m (thus n = 1, 25, 361, 5041, 70225, ... ).

[See R. Wainwright, in Puzzlers’ Tribute (A K Peters, 2002), 277–281; also Erich Friedman’s survey in erich-friedman.github.io/mathmagic/0607.xhtml.]

155. (a) Algorithm M finds 8 · 7571 solutions, in 60 Gμ.

(b) The maximum is 35 (not easy to find!), and the minimum is 5. [This exercise was suggested by Robert Reid, who found a minimum solution by hand in 2000.]

156. At level l of backtracking, branch on all ways to fill the leftmost unfilled cell of the topmost unfilled row. Even though no MRV heuristic is used, this method needs just 2.0 teramems (and negligible memory) to find 18656 solutions. The search tree has 61636037366 nodes.

We can save a factor of 8 by removing symmetry: The 1 × 1 square can be confined to cells (i, j) with i < 18 and j ≥ 35−i. Furthermore, if (i, j) is on the diagonal (j = 35−i), the context of the 1×1 square must be either or , and we can insist on the former. Now we find 2332 solutions (and 6975499717 nodes), in just 235 gigamems.

By contrast, the MCC problem (61) for n = 8 has 1304 items and 7367 options of total length 205753, when we restrict the options of #1 to i < 18 and j ≥ 35 − i. It needs 490.6 teramems to find 2566 solutions; postprocessing reduces that number to 2332, because 468 of those 2566 have #1 in position (i, j) with j = 35 − i.

We conclude that a dancing-links approach is decidedly not the method of choice for this partridge problem; straightforward backtracking with bitwise operations is more than 2000 times faster! Indeed, we might consider ourselves fortunate to pay “only” a 2000-fold cost penalty, since each of the 841 options for #8 in (61) contributes 65 nodes to doubly linked lists. Such updating and downdating keeps the dancers extremely busy.

[Historical notes: The 2332 solutions for n = 8 were first found by Bill Cutler in 1996, using a refinement of the backtrack approach described above. At that time no solutions for n < 11 had been known, although Wainwright knew how to solve 12 ≤ n ≤ 15 in 1981, and C. H. Jepsen and S. Ahearn had presented constructions for 11 ≤ n ≤ 33 in Crux Mathematicorum 19 (1993), 189–191. The puzzle can surely be solved for all n > 7, but no proof is yet known.]

157. Algorithm M readily shows the nonexistence of perfect packings, but the backtrack method of exercise 156 is much better to show that we can’t pack all but one 2 × 2. That method also shows that we can pack all but two of them:

158. The following solutions can be proved optimum with bitwise backtracking as in exercise 156:

159. Replace # by four primary items #0, #1, #2, #3 representing “quadrants,” and use #2⌊i/4⌋+⌊j/4⌋ in place of # in (64). Then partition into ten separate cases, in which the multiplicities m_0m_1m_2m_3 of #0#1#2#3 are respectively (2012, 2111, 2120, 3002, 3011, 3020, 3110, 4010, 4001, 5000). (Omit options containing #k of multiplicity 0.) These cases produce (134, 884, 33, 23, 34, 1, 16, 0, 22, 0) solutions, in (95, 348, 60, 23, 75, 8, 19, 2, 10, 0) megamems. (Notice that 4 · 134 + 4 · 884 + 8 · 33+4 · 23 + 8 · 34 + 8 · 1+4 · 16+8 · 0+4 · 22+4 · 0 = 4860.) The running time has decreased by a factor of 20.

[For larger values of n we could divide the cells into nine regions: eight octants, plus a special region containing the diagonals (and the middle row, column if n is odd).]

160. There are 589 components, among which are 388 isolated vertices and one giant of size 3804. The other 200 components have sizes ranging from 2 to 12. (For example, the first three solutions in (65) belong to the giant component; the other belongs to a component of size 8.)

161. In general, consider the problem of finding all the m-vertex dominating sets of a graph G; the n × n m-queen problem is the special case where G is the queen graph of order n. Then the options (64) have the form ‘# v v_1 ... vt’, where {v_1,...,vt} are the vertices adjacent to v, and # is a special primary item of multiplicity m.

Variant (i) is equivalent to asking for all kernels of size m (all of the maximal independent sets). Let there be a secondary item e for every edge in G; the options are then ‘# v v_1 ... vt e_1 ... et’, where ej is the edge between v and vj. An 8 × 8 chessboard has 8 · 91 = 728 kernels of size 5. (It also has 6912, 2456, and 92 kernels of sizes 6, 7, and 8; see exercise 7.1.4–241(a).)

For variant (ii) we simply shorten v’s option to ‘# v_1 ... vt’; some other option must then cover v. Exactly 352 of the 5-queen solutions satisfy (ii).

Variant (iii) seems a bit harder to formulate. Let there be a secondary item v^ for each vertex v. The option for choosing v can then be ‘#⁢ v⁢ v^:1v1.⁢ .⁢ .⁢ vt⁢ u^1⁢ : 0 . . . u^s :0’, where {u_1,...,us} = V \{v, v_1,...,vt} is the set of vertices not adjacent to v. The 8 × 8 chessboard has 20 clique-dominators of size 5.

[Chapter 10 of the classic work Mathematische Unterhaltungen und Spiele by W. Ahrens (1910) is an excellent survey of early work on queen-domination problems.]

162. Formulate these as MCC problems, by starting with the ordinary options for the n queens problem (see (23)), then adding additional options such as ‘# rj ck+1 aj+k+1 bj−k−1 rj+1 ck+3 aj+k+3 bj−k−3 rj+2 ck aj+k+2 bj−k+2 rj+3 ck+2 aj+k+5 bj−k+1’ to represent a contained Q_4, for 1 ≤ j, k ≤ n − 3. Here # is a new primary item, which is given the desired multiplicity.

1. (a) 15; one can, in fact, get disjoint Q_4 and Q_5 in a Q_15.

2. (b, c) 17. Put a queen in the center, make a pinwheel! [See Ahrens (1910), 258.]

3. (d) 22; see below. Algorithm M proves n = 21 impossible after 1.2 teramems.

4. (e) 16; there are four essentially different solutions.

5. (f) 19; see below. Only 35 Gμ to show that n = 18 is too small.

6. (g) 20(!). Once you know this, Algorithm X will find all 18 solutions in 2 Mμ.

7. (h) 22; there are 28 essentially different solutions.

8. (i) 25; see below. (After 6 teramems, we learn that n = 24 doesn’t work.)

163. Sometimes Algorithm M is called on to choose zero or more items from an empty list. Then it sets FT[l] ← i and xl ← i, where i is the item whose list is empty; but step M5 doesn’t actually tweak anything. The peculiar rule in (71) ensures that step M8 doesn’t actually untweak anything as we backtrack.

164. If xj ≤ N, node xj is the header for item xj; there’s no further option for such j.

[A good implementation will also extend answer 12, so that the relative positions of each xj in the search tree are identified. For this purpose one can add a new array SCORE, setting SCORE[l] ← θi and FT[l] ← 0 at the end of step M3. When printing the jth step xj of a solution, the old answer 12 is used if FT[j] = 0; otherwise that answer is modified as follows: If x ≤ N and (x = FT[j] or x = TOP(FT[j])), print ‘null NAME(x)’; otherwise print option x as before. Conclude by looping with i ← 0, q ← FT[j] rather than i ← TOP(x), q ← DLINK(i); report ‘k of SCORE[j]’ rather than ‘k of LEN(i)’.]

165. (a) To cover 2 of 4, we have 3 choices at the root, then 3 or 2 or 1 at the next level, hence (1, 3, 6) cases at levels (0, 1, 2). To cover 5 of 7, there are (1, 3, 6, 10, 15, 21) cases at levels (0, 1, ... , 5). Thus the search profile with item 1 first is (1, 3, 6, 6 · 3, 6 · 6, 6 · 10, 6 · 15, 6 · 21). The other way is better: (1, 3, 6, 10, 15, 21, 21 · 3, 21 · 6).

(b) With item 1 first the profile is (a_0, a_1, ... , ap, apa_1, ... , apaq), where aj=(j+dd). We should branch on item 2 first because ap+1 < apa_1, ap+2 < apa_2, ..., aq < apaq−p, aqa_1 < apaq−p+1, ... , aqap−1 < apaq−1. (These inequalities follow because the sequence 〈aj〉 is strongly log-concave: It satisfies the condition aj2>aj−1⁢ aj+1 for all j ≥ 1. See exercise MPR–125.)

166. (a) The “monus” operation x ∸ y = max(x − y, 0) is good for situations like this:

θp = (LEN(p) +1) ∸ (BOUND(p) ∸ SLACK(p)).

(b) It’s better to branch on p' (although this may be counterintuitive).

[The author’s implementation of step M3 breaks ties by first preferring an item with smaller SLACK, then preferring longer LEN when the SLACKs are equal. Thus, his MRV replaces answer 9 by this: Set θ ← ∞, p ← RLINK(0). While p ≠ 0, do the following: Set λ ← θp; if λ < θ or (λ = θ and SLACK(p) < SLACK(i)) or (λ = θ and SLACK(p) = SLACK(i) and LEN(p) > LEN(i)) set θ ← λ, i ← p; then set p ← RLINK(p).

167. Step M3 isn’t precisely defined; therefore any change to vp could possibly affect the behavior. But let’s assume that step M3 is implemented as in exercise 166.

Even so, there can be differences. A minor difference arises, for instance, if there are no options: A primary item with multiplicity [0 .. 1] will be inactivated by covering in step M4; with multiplicity [0 .. 2], it will become inactive at the end of step M5.

There can also be more significant differences. Suppose there’s just one option, ‘a’, and one primary item. If a has multiplicity 1, we simply cover a as in Algorithm X. But if a has multiplicity [1 .. 2], we’ll do some tweaking and untweaking — even entering a new level, and taking a null branch there.

On the other hand, the differences can’t get much worse. Let BOUND_0(p) and BOUND_1(p) denote the values of BOUND(p) when the upper bound vp has respectively been specified as Mp and Mp+δ. If the same options are chosen, we’ll have BOUND_1(p) = BOUND_0(p) + δ throughout the algorithm, because BOUND(p) is adjusted appropriately whenever the algorithm recursively reduces the problem by removing an option. Also SLACK_1(p) = SLACK_0(p) + δ. One can then prove, by induction on the computation, that the same options are indeed chosen (possibly with different amounts of tweaking).

Any two values of vp that are Mp + 2 or more will be totally equivalent.

168. Introduce a new primary item ‘!’ and a new secondary item ‘+’. Replace the two copies of α by ‘! +:0’, ‘! α’, ‘α +:1’. [Similarly, three copies of α can be replaced by ‘! +:0’, ‘! α’, ‘!! ++:0’, ‘!! α +:1’, ‘α ++:1’, after introducing ‘!!’ and ‘++’.]

169. Let there be one primary item, #, together with one secondary item for each vertex. And let there be one option, ‘# v v_1:0 ... vd:0’ for each vertex v, where v_1 through vd are the neighbors of v. Finally, let # have multiplicity t. [Notice that the secondary items in this construction are colored either with 0 or not at all!]

170. Introduce the primary item !v for each vertex, and give it d + 1 options: ‘# !v v:1 v_1:0 ... vd:0’, ‘!v v:0 v_1:1’, ‘!v v:0 v_1:0 v_2:1’, ..., ‘!v v:0 v_1:0 ... vd−1:0 vd:1’.

171. Let there be ten primary items v, for 0 ≤ v < 10; also fifteen primary items #uv, with multiplicity [1 .. 5], for each edge u −−− v, where the edges are 0 −−− 1 −−− 2 −−− 3 −−− 4 −−− 0, 0 −−− 5, 1 −−− 6, 2 −−− 7, 3 −−− 8, 4 −−− 9, and 5 −−− 7 −−− 9 −−− 6 −−− 8 −−− 5. Let there be 26 · 10 secondary items av through zv, for 0 ≤ v < 10; also 26 · 30 secondary items auv through zuv, for u―v; also a secondary item w for each word in, say, WORDS(1000). There are 26 options, ‘#uv au av’ through ‘#uv zu zv’, for each edge. And there are 10 options for each word; for example, the options for added are ‘v av:1 bv:0 cv:0 dv:1 ev:1 fv:0 ... zv:0 a_02 a_03 a_06 a_07 a_08 a_09 d_02 ... e_09 added’, where 0 ≤ v < 10.

Every solution to this MCC problem will be obtained 120 times, because the Petersen graph has 120 automorphisms. But symmetry can be broken by choosing the labels of 0, 1, and 3 at levels 0, 1, and 2, and by ordering the label ranks so that r_0 > r_1, r_0 > r_2, r_0 > r_3, r_0 > r_6, r_1 > r_4, r_1 > r_5, r_3 > r_7, r_3 > r_8, r_3 > r_9.

There are two solutions in WORDS(834), namely muddy, thumb, books, knock, ended, apply, fifth, grass, civil, (refer or fewer), found in 3.5 Tμ.

172. A construction analogous to answer 170 generates all solutions to the weaker problem where connectivity isn’t tested; it’s easy to remove the unconnected solutions from Algorithm M’s output. Consider cycles first: There are 1+(d2) options for each primary item !v, namely ‘!v v:0’ and ‘# !v v:1 v_1:a_1 ...vd:ad ’, where a_1 ...ad is a binary vector with a_1 + ··· + ad = 2. For the path problem, the options for the starting vertex should have a_1 + ··· + ad = 1, not 2. The options for all other vertices that aren’t adjacent to the starting vertex should have d additional options ‘# E !v v:1 v_1:a_1 ...vd:ad ’, with a_1+···+ad = 1, where E is a new primary item signifying the end vertex.

(a) Paths of length l are obtained when the multiplicity of # is set to l +1.

First let’s restrict consideration to paths that start in the corner cell (0, 0). Then every essentially distinct path occurs twice — reflected about the diagonal. (i) There are 16 distinct snake-in-the-box king paths of length 31 from a given corner, found in 6 Tμ. One of them, illustrated below, also ends at a corner; hence it occurs four times, not two — twice in each direction. These paths are optimum, because we can divide the board into sixteen 2 × 2 subsquares, each of which can contain at most two kings. (ii) A single run, with the multiplicity of # set to [32 .. 33], suffices to find the 13 distinct knight solutions of length 31 in 58 Gμ, simultaneously showing that length 32 is impossible. One of the most remarkable solutions is shown below. (iii) With bishops we should first eliminate all squares of the wrong parity, because they cannot be connected to the start. Then the 32 solutions of length 12 are found in just 13 Mμ. (It’s not difficult to prove by hand that an n×n board has exactly 2n−3 bishop solutions of length 2n − 4, when n is even.) (iv) Rook solutions are even easier to enumerate by hand: There are (n − 1)!2 of them, because we always have n − k choices at steps 2k − 1 and 2k. (Algorithm M finds the 7!2 = 25401600 solutions in 625 Gμ, while generating also 21488110 disconnected impostors.) However, (n − 2)!2 − (n − 2)! of those solutions are counted twice, because they go from corner to corner and have no symmetry. Hence there are 25401600 − 517680/2 = 25142760 distinct rook solutions of length 14. (v) Finally, there are 134 distinct queen solutions of length 11 — found and proved optimum in 17 Gμ, despite having 16788 options of total length 454380(!). The unique solution that occupies opposite corners is shown here. (You may enjoy finding another unique 11-step path, which begins slowly by moving just one diagonal step.)

Now let’s consider paths that start in cell (0, 1) and do not end in a corner. (i) Five solutions with 32 kings are found (in 3.7 Tμ); but they all have 3-cycles and are disconnected. (ii) Knights, however, yield a big surprise: There’s a unique path of length 33, doubly counted! (Found in 43 Gμ.) (iii) Bishop paths can’t have length 12 unless they start or end in a corner. (iv) There are N = (n − 1)!2 − 2(n − 2)!2 solutions where the rook first moves down, and N where it first moves sideways. Of these, 2Nc end at (n − 1,n − 2) and are double-counted by central symmetry, where Nc=(2⌊n/2⌋−1(⌊n/2⌋−1)!)2; Nt = 2(n−2)! end at (1, 0) and are not double-counted by transposition; Nt end at (n − 2,n − 1) and aren’t double-counted by dual transposition. So there are 2N − Nc − (2(n − 2)!2 − Nt) = 47691936 equivalence classes when n = 8. (v) Another nice surprise greets us, namely a unique queen path of length 12!

The next step is to consider paths that start in (0, 2) and don’t end in the 12 types of cells already considered. And so on, for seven more cases. Of course rook counting gets hairier and hairier; we shall omit it. Unexpectedly, there’s also another maximum queen path(!). All of these computations are fast, except that the kings need 6.3 Tμ.

(b) Cycles are similar, but symmetry now becomes even trickier. (i) The six distinct 31-cycles of a king are asymmetric, so they each appear eight times when reflected and/or rotated. (ii) But the four distinct 32-cycles of a knight include two that are equivalent to their transpose, and one (shown below) with central symmetry. (iii,v) A bishop has 36 distinct 12-cycles, and a queen has five 13-cycles, all asymmetric.

(iv) A rook, on the other hand, has oodles of 16-cycles, some of which (like the one illustrated) even have 4-fold symmetry under both horizontal and vertical reflection. Every rook snake-in-the-box 16-cycle can be represented uniquely as (p_0q_0 p_0q_1 p_1q_1 p_1q_2 ... p_7q_7 p_7q_0), where p_0p_1 ...p_7 and q_0q_1 ...q_7 are permutations of {0, 1,..., 7} with p_0 = 0 and q_0 < q_1. Consequently there are 8!2/16 = 101606400 of them, if symmetry isn’t taken into account. That cycle is equivalent to its transpose if and only if pj = q(k−j)mod 8 for some k and all j; there are 8!/2 = 20160 such cases. It is equivalent to its 180° rotation if and only if pj + p4+j = qj + q4+j = 7 for 0 ≤ j < 4; there are 6 · 4 · 2 · 8 · 6 · 4 · 2/2 = 9216 such cases. And it is equivalent to both, in 6 · 4 · 2 · 8/2 = 192 cases. Hence by “Burnside’s lemma” there are (101606400 + 0 + 9216+0+ 20160 + 0 + 20160 + 0)/8 = 12706992 equivalence classes of rook cycles.

[T. R. Dawson introduced this problem for knights, and presented an example path of length 31 and an example cycle of length 32, in L’Echiquier (2) 2 (1930), 1085; 3 (1931), 1150. C. C. Verbeek posed the problem of maximizing the number of queens such that each is “attacked by exactly two others” in Elsevier’s Weekly (June 1971); if we allow several queens in the same row, arguing that the first doesn’t attack the third, 14 queens are actually possible (see P. Torbijn, Cubism For Fun 17 (1991), 19). The name ‘snake-in-the-box’ was coined by W. H. Kautz, IRE Trans. EC-7 (1958), 177–180, for the case where G is an n-cube. The term ‘coil-in-the-box’ is often used nowadays for a snake-in-the-box cycle.]

Nikolai Beluhov proved in 2018 that, if n ≥ 6 is even, all snake-in-the-box king paths of the maximal length n^2/2 − 1 on n × n boards have an interesting structure, which can be characterized completely. In fact, he showed that exactly 2n+(n mod 4)/2 such paths are distinct under symmetry. Furthermore, there are exactly six distinct snake-in-the-box king cycles of length n^2/2 − 1, when n ≥ 8 is a multiple of 4.

With arguments of a different kind, Beluhov has also proved that the longest snake-in-the-box paths and cycles of a knight, on an m × n board, have length mn/2 − O(m + n). [To appear.]

173. (a) Write ‘k −−− ij’ if clue k is a (knight or bishop) move away from cell (i, j). For each row, column, and box, compute “quotas” ri, cj, and bx, equal to 3 minus the number of pieces already present among the given clues. Also compute the quota pk for each clue k, equal to the label minus the number of neighboring cells already occupied. There is no solution if any quota is negative.

Say that cell (i, j) of box x is known if it is occupied, or if ri = 0 or cj = 0 or bx = 0, or if pk = 0 for some k −−− ij. Introduce primary items Ri, Cj, Bx, Pk for each row, column, box, or clue with a positive quota, having multiplicities ri, cj, bx, pk. There is one option for each unknown cell, namely ‘Ri Cj Bx ∪{Pk | k −−− ij}’.

(b, c, d) See Fig. A–3. The knight puzzles with labels ≥ 6, and the bishop puzzles with labels 0, 10, and 12, are due to N. Beluhov; the others represent the author’s best early attempts, not necessarily minimum. Solutions can be found in Appendix E.

Fig. A–3. A gallery of knight and bishop sudoku puzzles.

[These variants of sudoku were devised by David Nacin and first published in MAA Focus 38, 6 (Dec. 2018/Jan. 2019), 36; see also quadratablog.blogspot.com.]

174. Beluhov’s remarkable solution, which he obtained with the help of a SAT solver, is also a pair of “rainbow puzzles” — every possible knight label occurs exactly once(!):

[Also shown are his 10-clue puzzles in which all the labels are equal.]

175. We can allow an option α to be repeated twice by simply replacing it by three options ‘α x’, ‘# x’, ‘# α’, where # is a new primary item and x is a new secondary item. (If α contains uncolored secondary items y_1, ... , ys, we should first replace them by y_1:c, ... , ys:c, where c is a new color.)

In general if α is the ith option and if ai = a +1 > 1, replace α by the 2a +1 options ‘α x_1i’, ‘#1i x_1i’, ‘#1i α x_2i’, ‘#2i x_2i’, ‘#2i α x_3i’, ... , ‘#ai xai’, ‘#ai α’, where #ti and xti are new primary and secondary items.

176. (a) Introduce 3N items {Aj,Bj, #j | 1 ≤ j ≤ N}, to be used in M options {Aj | aij ≥ 1}∪ {Bj | aij = 2} for 1 ≤ i ≤ M. (For example, the option for row (2, 1, 0, 2, 0, ... ) would be ‘A_1 B_1 A_2 A_4 B_4’.) Add 2N further options ‘#j Aj’, ‘#j Bj’ for 1 ≤ j ≤ N. Use Algorithm M with multiplicities (2, 1, 1) for (Aj, Bj, #j).

(b) The same construction works, but with multiplicities (3, 1, 1).

(c) Now use 4N primary items {Aj,Bj,#j,#j′} and N secondary items xj. Change the 2N special options to ‘#j⁢  Aj’,‘#j⁢  Bj⁢  xj’,‘#j′⁢  Aj⁢ xj’⁢,‘#j′⁢  Bj’, for 1 ≤ j ≤ N. Use multiplicities (4, 2, 1, 1).

(d) With 7N primary items {Aj,Bj, #1j,..., #5j} and 4N secondary items {x_1j,x_2j,x_3j,x_4j}, the special options are ‘#1j Aj’, ‘#1j Bj x_1j’, ‘#2j Aj x_1j’, ‘#2j Bj x_2j’, ... , ‘#5j Aj x_4j’, ‘#5j Bj’, and the multiplicities are (11, 5, 1, 1, 1, 1, 1).

177. (a) The 2s3t − 1 nonzero vectors a_1 ...asb_1 ...bt with 0 ≤ ai ≤ 1 and 0 ≤ bi ≤ 2 form the rows of a matrix A. Allow the 2t − 1 rows with ai = 0 and bi ≠ 2 to be repeated, via answer 175; also encode the 2’s via answer 176. That leads to s+3t+2t−1 primary items, 2t − 1 secondary items, and a total of 2s3t − 1+2t + 2(2t − 1) options. (There are 91914202 multipartitions when s = t = 5. Algorithm M generates them at a rate of about 1300 mems per solution; that’s only about seven times slower than the special-purpose Algorithm 7.2.1.5M.)

(b) This problem is easier, because we simply disallow using an option twice. That leaves us with s +3t primary items and 2s3t − 1+2t options.

(Exercise 7.2.1.5–73 enumerates the number of solutions P (s, t) for part (a). The same argument gives a similar recurrence for the number Q(s, t) of solutions to part (b):

Q(s,0)=ϖs;2Q(s,t+1)=Q(s+2,t)+Q(s+1,t)−∑K(nk)Q(s,k).

With this formula one finds quickly, for example, that Q(5, 5) = 75114998.)

178. (a) Since 360 = 2^3 · 3^2 · 5, we need first to extend exercise 176 to matrices of 0s, 1s, 2s, and 3s. Encoding aij = 3 in option i can be done by using items Aj, Bj, Cj. To ensure a total of 3 in that column, let #j and #j′ be new primary items, and give multiplicity (3, 1, 1, 1, 1) to (Aj,Bj,Cj,#j,#j′); also let xj be secondary. Then the special options ‘#j⁢  Aj’, ‘#j⁢  Bj⁢ xj’,‘#j′⁢ Aj⁢ xj’,‘#j′⁢ |Cj’ will fix everything up.

This makes an MCC problem with 29 options, 9 + 1 items, and 34 solutions.

(b) Now use exercise 175 to allow the options for factors 3 and 2×3 to be repeated at most twice, and to allow the option for factor 2 to be repeated at most thrice. The MCC problem now has 37 options, 13+5 items, and 52 solutions. [These solutions were first studied by John Wallis; see exercise 7.2.1.7–28.]

179. From 1000 + 0110 + 0001 we get four solutions 100000 + {011100, 011100} + {000011, 000011}; from 1110 + 0001 we get two solutions 111100 + {000011, 000011}; and from 1010 + 0101 we get 101000 + 010111.

180. The text showed that o_1 = ‘i_1’ and that i_2 and o_5 exist, when t = 4 and t′ ≥ 1. Continuing that example, if s_2 = 5 so that t′ ≥ 2, then option o_2 intersects only {o_1,...,o_5}; hence o_2 = ‘i_1 i_2’, and i_2 cannot occur in more than 4 options. Its appearances must therefore be in {o_2,o_3,o_4,o_5}.

Furthermore, o_3 must be ‘i_1 i_2 i_3 ... ’ for some third item, i_3, since we can’t have o_3 = o_2. Consequently there’s an option o_6 = ‘i_2 i_3 ... ’. And so on.

181. (c_0,c_1,c_2,c_3,c_4) = (188, 248, 320, 425, 566)/96, by the initial values in the text.

182. (To establish the lower bound in Theorem E, make n copies of this problem, on disjoint four-tuples of items. This yields 7n solutions, in a search tree with (5 · 7n − 3)/2 nodes. Notice that the branching factor never exceeds 3 in this construction.)

183. (Can one, for example, often make the branching factor t = 4?)

184. Yes. If we can write t = an−1 ϖn−1 + an−2 ϖn−2 + ··· + a_0 ϖ_0, with 0≤aj≤(n−1j) for 0 ≤ j < n, we get such a problem by letting the options consist of (i) all 2n−1 −1 subsets of {1,...,n−1}; (ii) exactly aj subsets of {1,...,n} of size n−j that contain n.

To write t in that form, suppose t=(n−1n−1)  ϖn−1+⋅⋅⋅+(n−1n−k+1)  ϖn−k+1+an−k⁢ ϖn−k+t′, where 0≤an−k<(n−1n−k) and 0 ≤ t′ <  ϖn−k. Then, by induction, we can write t′ = an−k−1 ϖn−1−k + ··· + a_0 ϖ_0, with 0≤aj≤(n−k−1j)≤(n−1j).

For example, 10000 = 1·4140+6·877+(1·203+7·52+2·15+(0·5+(0·2+(1·1)))).

185. We get the most solutions when we have the most options, namely the 2N_1+N_2 − 2N_2 subsets that aren’t entirely secondary. Then the solutions are the set partitions that include at most one entirely secondary block; and the number of such set partitions is seen to be ∑k{N1m}(m=1)N2, when we consider their restricted growth strings.

186. (a) The list for i consists of all 2n−1 subsets that contain i. So there are (n−1k−1) operations hide(p) on options p of size k; and un=1+∑k(n−1k−1)(k−1)=(n−1) 2n−2+1.

(b) The lists get shorter, so the algorithm does un−1 + ··· + un−(k−1) updates.

(c) Sum un+∑k(n−1k−1)(sn−1−sn−k), where sn=∑k=1nuk=(n−2)2n−1+n+1.

For example, (v_0,v_1,...,v_5) = (0, 1, 3, 12, 57, 294); (x_0,x_1,...,x_5) = (0, 1, 4, 18, 90, 484).

187. (a) We have X′(z) = Σn xn+1zn/n! = V′(z)+ezX(z), where V (z) = Σn vnzn/n!. The given function solves this differential equation and has X(0) = 0.

(b) Similarly, we have Tr,s′(z)=ez⁢ Tr,s(z)+zr and Tr,s(0) = 0.

(c) Integrate by parts.

(d) For example, T1,3(z) = 4eez−1 + 2T_0,0(z) − ze2z − (2z +1)ez − 2z − 3, by (c).

188. By induction, ϖ^nk is the number of n-element, single-tail set partitions (equivalence relations) for which n > 1 and 1 ≢ 2, ... , 1 ≢ k. (For example, if we know that 22 single-tail partitions of {1, 2, 3, 4, 5} have 1 ≢ 2, and that 6 such partitions of {1, 2, 3, 4} have 1 ≢ 2, then 6 single-tail partitions of {1, 2, 3, 4, 5} must have 1 ≢ 2 and 1 ≢ 3; hence 16 of them have 1 ≢ 2 and 1 ≢ 3.) Therefore ϖ^nn=ϖ^n−1, for all n ≥ 1.

[Leo Moser played with this triangular array in 1968 and found the generating function ∑nϖ^n⁢ zn/n! = eez∫0z  e−etdt; he showed his results to R. K. Guy, who told N. J. A. Sloane; see OEIS sequences A046936 and A298804. If we start with ‘0, 0, 1’ on the diagonal instead of ‘0, 1’, we get Gould’s 〈an_2〉 = 〈 0, 0, 1, 1, 4, 14, 54, 233,... 〉; etc.]

189. (a) |eez| = |ee x cos y+iex sin y | = exp(ex cos y); |e−ez | = exp(e−x cos y).

(b) |∫0θ exp (−eξeiϕ)⁢ d(ξeiϕ)+∫ξ∞e−etdt|=O(ξ⁢ exp(−e⁢ cosy))+O(exp(−eξ)) ; |eez | = O(exp(eξ)); and we have x=ξ⁢  cosθ≥ξ−2.25/ξ,cos y≥cos32.

(c) We have ∫z∞e−etdt=∫z∞e−etdt−∫z1e−euzd(uz)=g^/3−I. Let max |eeuz| for 0 ≤ u ≤ 1 be exp(−eu_0 x cos u_0y). If cos u_0y ≥ 0 we have |I| = O(ξ). Otherwise if cos y − cos u_0y ≤ 1 we have |eezI| ≤ ξ exp(ex cos y − eu_0x cos u_0y) ≤ ξ exp(ex cos y − ex cos u_0y) ≤ ξ exp(ex). Otherwise we use a more delicate argument: Since cos(a − b) − cos(a + b) = 2(sin a)(sin b), we have |sin u0−12y|=12|(cos y−cos u0y)/sinu0+12y|≥12, hence u_0 ≤ 1 − π/(3y). And in this range, u0⁢ x⁢≤x−π3x/y=ξ cosθ−π3cot θ≤ξ−cξ1/3+O(1), where c3=38π2.

The desired bound now holds in each case because x=ξ1−sin2θ≤ξ−9/(8ξ).

(d) If π2≤θ≤π, |eez| exp(−eu_0x cos u_0y) = O(1). Since pn−1/(n−1)!=12πi∮R(z)dz/zn, and since ϖn−1/(n−1)!=Θ(eeξ/(ξn−1ξn)) by 7.2.1.5–(26), we have |pn−1/ϖn−1|=O(ξn⁢⁢ exp(−c2⁢ eξ/ξ)) for all c2<98. And −eξ/ξ = −n/ξ^2 < −n/ln^2n.

[These results, and considerably more, were proved by W. Asakly, A. Blecher, C. Brennan, A. Knopfmacher, T. Mansour, and S. Wagner, J. Math. Analysis and Applic. 416 (2014), 672–682. In particular, they proved that ank/ϖn rapidly approaches the constant g^k=∫0∞  tk−1e1−etdt/k!=∫0∞e−x⁢  lnk(1+x)dx/k!, for all k > 0.]

Historical notes: Leonhard Euler computed the constant ĝ when he argued that this value can be assigned to the divergent series ∑n=0∞(−1)nn! [Novi Comment. Acad. Sci. Pet. 5 (1754), 205–237]. Benjamin Gompertz, who did not know the constant ĝ explicitly, studied the probability distributions F (x) = 1 − a1−bx for a, b > 0 and x ≥ 0 [Philos. Trans. 115 (1825), 513–585]. His name came to be associated with ĝ because, for example, a random variable with a = e in his distribution has E X = ĝ/ln b.

190. Empirically, these signs are essentially periodic, but with a slowly increasing period length as n grows. For example, the signs for 4000 ≤ n ≤ 4100 are +2−4+4−5+4−4+5−4+4−5+4−4+4−5+4−4+5−4+4−5+4−4+4−5. The quantities ϖ^nk−g^ϖnk for 1 ≤ k ≤ n ≤ 100 have the interesting sign pattern shown at the right. (See exercise 188.) Complex variables are evidently interacting here somehow!

191. The mean is G′(1) = 1+ ĝ; the variance is G″(1)+G′(1)−G′(1)2 = 2ĝ_2 + ĝ− ĝ^2 ≈ 0.773. [Incidentally, G(z) can also be written eΓ(1+z)−∑k=1∞(−1)kez/((k+z)k!).]

192. Let ξeξ = n as in 7.2.1.5–(24). Then, when x = eξ − 1+ t and t is small, we have e−x(ln(1 + x))n ≈ A exp(−(1 + ξ)t^2/(2n)), where A = exp(n ln ξ + 1 − eξ). Trading tails and integrating over −∞ < t < ∞ gives g^n⁢ ~ A⁢ 2πn/(1+ξ)/n!.

193. At level 0, when given the complete graph Kt+1, the algorithm does t + 1 updates when covering i in step X4, and t updates when covering each of t values of j in step X5. Thus U(t +1) = 1+ t + t^2 + tU(t − 1).

194. (a) In general we have X(2q +1) = (2q)(2q −2) ... (2) (a_0 + a_2/2+a_4/(2·4)+···+) a2q/(2·4·. . .·(2q))) = 2qq!S −R, where S = Σn ≥ 0 a2n/(2nn!) and R = a_2q+2/(2q+2)+ a_2q+4/((2q+2) · (2q+4)) + ··· . Hence when at = 1 we have S = e1/2 and 0 < R < 1. [This result was noticed in 1999 by Michael Somos; see OEIS A010844.]

(b) In general, X(2q) = ((2q)!/(2qq!))S − R, where S = X(0) + a_1 + a_3/3 + a_5/(3 · 5) + a_7/(3 · 5 · 7) + ··· and R = a_2q+1/(2q+1) + a_2q+3/((2q+1) · (2q+3)) + ···. When at=X(0)=1,  S−1=1+1+1/3+1/(3⋅5)+e1/2⁢ erf(1/2)/((12)1/2/(12!)), and 0 < R < 1. So the answer is ⌊(1+eπ/2⁢ erf(1/2))(2q)!/(2qq!)⌋.

(c) 2qq!C − 2q + O(1), where C = Σn ≥ 0 (1+2n + 4n^2)/(2nn!) = 5e1/2 ≈ 8.24361.

(d) ((2q)!/(2qq!))C′ − 2q + O(1), where C′=3+5eπ/2⁢ ⁢  erf(1/2) ≈10.05343.

195. Assume that q, r > 1, and let v be the unique vertex of degree 2. The algorithm will try to match v with the vertex at its left; that leaves a problem of matching the independent graphs K2q and K2r. If q ≤ r, each matching of K2q will initiate a computation of the matchings of K2r; otherwise each matching of K2r will initiate the matchings of K2q. So the running time of this phase will be C′ updates per solution, where C′ is the constant of answer 194(d) and there are (2q)!(2r)!/(2qq!2rr!) solutions.

The algorithm will also try to match v with the vertex at its right. That leaves a problem of independently matching K2q+1 and K2r−1, and there are no solutions. The running time of this phase will be C times min(2qq!, 2r−1(r − 1)!), where C is the constant of answer 194(c). (Curiously, it’s actually negligible compared to the other phase.)

196. (a) b_1 ...b_9 = 135778899. (Draw the bipartite graph, and rotate it 180°.)

(b) Let k¯=n+1−k for 1 ≤ k ≤ n. Then ‘Xj Yk’ is a dual option if and only if ‘Yj¯⁢  Xk¯’ is an original option; q_1 ...qn is the inverse of an original solution if and only if q¯n .⁢ .⁢ .⁢ ⁢q¯1  is a dual solution.

(c) 1 + a_1(n + 1), because each Yk for 1 ≤ k ≤ a_1 appears in n options.

(d) a_1(a_2 − 1)(a_3 − 2) ... (an − n + 1). [This number must therefore be equal to b_1(b_2 − 1)(b_3 − 2) ... (bn − n + 1) — and that’s not an obvious fact!]

(e) Let Πj=∏i=1j(ai−i+1). From (c), the answer is 1+(∑j=1n(n+3−j)Πj)−Πn.

(f) 1+(∑j=1n(n+3−j)nj)−n!≈(4e−1)n!. [Perfect matchings of Kn,n.]

(g) 6 · 2n − 2n − 7, because Πj = 2j for 1 ≤ j < n, and Πn = 2n−1.

(h) Now ∏n=⌊n+12⌋  ⌊n+22⌋; and the total number of updates, divided by Πn, is therefore 6 + 4/1! + 5/2! + ··· + O(n^2/(n/2)!) ≈ 4e − 1.

(i) If b_1 < a_1, the first branch is on Yn, not X_1; and 1+b_1(n+1) updates are made at root level. (The example problem in (a) branches on Y_9, then X_2, then Y_8, etc.)

197. (a, b). Induction; σst can in fact be any permutation that takes s ↦ t and doesn’t increase any other element.

(c) C(a1,. . . ,  an)=∏j=1n(z+aj−j), by (a), since we gain a cycle in that product representation if and only if tj = j. I(a1,. . . ,  an)=∏j=1n(1+z+⋅⋅⋅+zaj−j), by (b). [See exercise 7.2.1.5–29; also M. Dworkin, J. Combinatorial Theory B71 (1997), 17–53.]

(12120000000181814141400001818141414000011211216161613000124124112112112161200172172136136136118161313172172136136136118161313172172136136136118161313172172136136136118161313)

198. (a) If s > ar we have πrs = 0. Otherwise let q be the smallest j with aj ≥ s; then q ≤ r. Each permutation of P (a_1,...,an) with pr = s corresponds to one of P(a1′,.⁢ .⁢ .⁢ ,ar−1′, ar+1′,.⁢ .⁢ .⁢ ,an′ ), where aj′=aj−[j≥q]. Thus (ar+1−r)πrs=∏j=qr−1(aj−j)/(aj+1−j).

(b) We have q′ ≥ q when s′ > s. Consequently πrs/πrs′=∏j=qq′−1(aj−j)/(aj+1−j) for all r ≥ q′, if πrs′ > 0. [In such cases the parameters r and s are said to be “quasi-independent.”]

199. Assume by symmetry that m ≤  ⌈n/2⌉. With the MRV heuristic it’s not difficult to see that every branch at level l for l < m is on some ai for i ≤ m, with exactly (n − l)(m − 1 − l) descendants. Hence there are nl(m−1)l nodes on level l. The total number of nodes when m ≈ n/2 is huge, Θ((n − 2)!); and there are no solutions.

200. (a) When all n^3 options are present, det Q(X) = Σ sign(p)v1p_1 q_1 ... vnpn qn, summed over all permutations p = p_1 ... pn and all n-tuples q = q_1 ... qn with qj ∉ X. Summing (−1)|X| det Q(X) yields Σ sign(p)v1p_1 q_1 ...vnpn qn where both p and q are permutations. (This is essentially an application of the inclusion-exclusion principle.) Set vijk ← 0 if option ‘ai bj ck’ isn’t present.

(b) Assign a random integer in [0 .. p) to each of the M given options, where p is a prime greater than 2M, and evaluate s = S mod p. If s ≠ 0, S is nonzero. If s = 0, S is nonzero with probability less than 1 − (1−1/p)M < M/p < 1/2, by exercise 4.6.1–16, because S is linear in each variable. Repeating r times will fail with probability < 2−r.

[In practice, 2n is often an overestimate because many of the determinants are obviously zero. For example, if Q(X) has an all-zero row or column, so does Q(X′) for all X′ ⊇ X. This method shines on unsolvable examples such as those of exercise 199. Björklund’s paper, STACS 27 (2010), 95–106, has more general results.]

201. (a) “Given n people seated at a circular table, how many seating arrangements do not require anybody to move more than one place left or right?”

(b) Two solutions in which everybody moves, plus Ln solutions (a Lucas number) in which at least one person remains in the same seat.

(c) An interesting recursive structure leads to the answer 5Ln+2 + 10n − 33. [This analysis depends on using the given ordering to break ties in step X3 when several lists have the minimum length.]

202.

203. (a) Yes; T ⊕ T′ ⊕ T″ is the search tree corresponding to A ⊕ A′ ⊕ A″.

(b) No; .

204. By definition of T ⊕ T′, we have subtree(αα′) = subtree(α) ⊕ subtree(α′). Hence deg(αα′) = min(deg(α), deg(α′)).

Let ancestors(α) = {α_0,...,αl} and ancestors(α′)={α0′,.⁢.⁢.⁢,αl′′}. Suppose αα′ is dominant in T ⊕ T′ and deg(αα′) = d. If 0 ≤ k < l, some ancestor αk ′αk′ of αα′ has deg(αk) = deg(αkα′k′) < d; hence α is dominant. Similarly, α′ is dominant.

We’ve proved the “only if” part, but the converse is false: .

205. The first statement follows easily from the definition (see exercise 202). Suppose αα′=αlαl′′∈T⊕T′, as in answer 204, where neither α nor α′ is dominant, and where l + l′ is minimum. Then l > 0 and l′ > 0, because α_0 and α0′ are dominant.

Assume that the parent of αα′ is ααl′−1′. Then αl′−1′ is dominant, and αl isn’t. So there’s a k < l such that deg(αk) = max(deg(α_0),..., deg(αl)). Hence there’s a maximum k′ < l′ such that αkαk′′ is an ancestor of αα′. Then deg(αk′′)≤deg(αl′−1′) < deg(α), and αkαk′+1′ is also an ancestor. But αkαl′′ isn’t. Contradiction.

A similar contradiction arises when the parent of αα′ is αl−1 α′.

206. Replace each solution node of T by a copy of T′.

207. (a) If λj = 4 we now prefer the 5-way branch on i, because λi′=7/2<11/3=λi′. If λj = 3 we prefer min(i, j), because λi′=3=λj′. If λj = 2 we still prefer the binary branch on j to the ternary branch on i. And if λj = 1 or 0 we certainly prefer j.

(b) Include two new fields, ACT and STAMP, initially zero, in each item node. (They can share an octabyte, if ACT is a short float and STAMP is a tetrabyte.) A global variable TIME serves as the “convenient clock.” Another global, BUMP (which is a short float, initially 10−32), is the amount by which we advance activity scores. Whenever i is covered or uncovered, or whenever LEN(i) is changed, we check to see if STAMP(i) = TIME; if not, we set ACT(i) ← ACT(i) + BUMP and STAMP(i) ← TIME.

The “clock” advances at the beginning of steps X4, X5, X6, and X7. This means that TIME ← (TIME +1)mod2^32 and BUMP ← BUMP/ρ. (Furthermore, if BUMP ≥ 10^29, we divide BUMP and all ACT fields by 10^64, to avoid overflow. We limit ρ to be at most .999, so that each αi is at most 1000.)

These changes allow us to replace the definition of λ in step X3 (answer 9) by λ ← (LEN(p) ≤ 1? LEN(p): 1+ LEN(p)/(1+ μ ACT(p)/BUMP)).

(c) Consider (90) first. After branching on 00 and trying option ‘00 01’, we have α_00 = α_02 = ρ, α_01 = 1+ ρ, α_04 = α_05 = α_06 = 1, and the other α’s are zero. We want λ05′=1=3/(1+μα05) to be less than λ10′=1+2; that is, μ > 1/2. Later, after trying option ‘00 02’, we’ll have α_05 > 1 and α_06 > 1; again, item 01 isn’t chosen.

Problem (92) is trickier. After trying ‘00 01’, the nonzero α’s are α_00 = α_02 = ρ, α_01 = 1 + ρ, and α_03 = α_04 = α_05 = 1. We’ll prefer the 3-way branch on 02 to the 2-way branch on 20 if μ > 1/(2ρ); and we’ll even prefer the 4-way branch on 04 (or 05) to that 2-way branch, if μ > 1. In either case we’ll reach a solution to problem 0 before starting on problem 1. The same calculations then take us to problem 2 only when problem 1 has been solved; etc. (Furthermore, when coming back down there will be no incentive to go back up. In fact, 4-way branches will be done on the items k3 because of their high activity scores.)

(d) The normal Algorithm X finds all 212 solutions in 92 Gμ, with a 54-meganode search tree. This modification finds them in 51 Gμ, if we set μ = 1/8 and ρ = .99, with a 26-meganode search tree. (With μ = 1/2 and ρ = .9, the time is 62 Gμ. In long runs, the α scores tend to approach 1/(1 − ρ); so increases in ρ usually imply decreases in μ.)

208. The original problem has primary items ij for 0 ≤ i, j ≤ e, and eight kinds of options ‘{ij + δ | δ ∈ Sk}’ for all cells ij + δ that are in range, where S_0 = {01, 11, 21, 31, 10}, S_1 = {00, 01, 02, 03, 11}, S_2 = {00, 10, 20, 30, 21}, S_3 = {10, 11, 12, 13, 02}, S_4 = {01, 11, 21, 31, 20}, S_5 = {00, 01, 02, 03, 12}, S_6 = {00, 10, 20, 30, 11}, S_7 = {10, 11, 12, 13, 01}. Options that involve the center cell 77 come only from S_0.

The modified problem adds secondary items Vij and Hji, for 0 ≤ i ≤ b, 1 ≤ j ≤ d. It inserts Vij, H(i+1)j, Vi(j+1), Hij respectively into the options with S_4, S_5, S_6, S_7.

(The 16 solutions to this problem represent 2^2 +2^4 +2^5 +2^2 +2^3 +2^2 +2^5 + 2^3 +2^5 +2^3 +2^2 +2^4 +2^3 +2^4 +2^2 +2^4 = 212 solutions to the original. We’re lucky that none of those solutions has an ‘H’ that includes 77.)

209. With the modified options ‘0 1 A’, ‘0 2 B’, ‘1 4 5 B’, ‘2 3 4 A’, obtained from the bipairs (‘0 1’, ‘2 3 4’; ‘0 2’, ‘1 3 4’) and (‘0 1’, ‘2 4 5’; ‘0 2’, ‘1 4 5’), we get the balanced search tree shown here.

210. Add a new primary item #A and give it multiplicity [0 .. 2]. Insert it into options α′, β′, γ′. Then use the nonsharp preference variant of Algorithm M.

211. No bipairs. (But Langford has bitriples, and all three have “biquadruples.”)

212. (a) Order the options first by their smallest item, and secondly by lexicographic order among those with the same smallest item.

(b) Yes. For example, we can let 1 < 2, and 1 < 4 < 0 < 5.

213. Yes, provided that we regard a proper prefix of a string as lexicographically larger than that string (contrary to the conventions of a dictionary). Otherwise the condition fails when α is a prefix of α′ (although exercise 212 remains valid).

Suppose the items of α and β are respectively represented by the digits j and k in rgs(π), the restricted growth string of π. Then j will also represent α′ in rgs(π′), and both strings will be equal up to the point where j first appears.

Let β′ be represented by k′ in rgs(π′); then k′ > j. Consider the leftmost place where rgs(π) differs from rgs(π′). If that digit is j in rgs(π), it is k′ in rgs(π′). Otherwise it is k in rgs(π); but then it is j in rgs(π′), and α is a prefix of α′.

214. We can find all solutions Σ that reduce to a given strong solution Σ_0, by repeatedly reversing the construction in the proof of Theorem S — replacing joint occurrences of α and β by joint occurrences of α′ and β′, for all canonical bipairs, in all possible ways. (It’s a reachability problem: to find all nodes of an acyclic digraph, given the sinks.)

Notice that different strong solutions can lead to the same nonstrong solution. For example, in the 2DM problem with options {xX, xY, yX, yY, yZ, zY, zZ}, where uv stands for ‘u v’, we might have the canonical bipairs (yX, xY; yY, xX), (yZ, zY; yY, zZ). The strong solutions {xY, yX, zZ} and {xX, yZ, zY} both lead to the nonstrong {xX, yY, zZ}. (However, in that same problem, we could have made the bipairs (yX, xY; yY, xX), (yY, zZ; yZ, zY) canonical. Then there would have been only one strong solution.)

215. (a) This is the number of 4-cycles, of which there are 3(2q+14): Four vertices i < j < k < l can form three 4-cycles, with either j or k or l opposite i.

(b) For convenience, denote options by ij instead of ‘i j’. If i < j < k < l, we exclude (i, j, k, l) unless min(ij, ik, il, jk, jl, kl) = ij or kl. We exclude (i, k, j, l) unless min(ij, ik, il, jk, jl, kl) = ik or jl. We exclude (i, l, j, k) unless min(ij, ik, il, jk, jl, kl) = il or jk. Hence exactly two of the three possibilities are excluded.

(c) When i < j < k < l they are (i, k, j, l) and (i, l, j, k).

(d) The root has 2q children, branching on 0. All of them are leaves except for the branch ‘0 1’. That one has 2q − 2 children, all of which are leaves except for the branch ‘2 3’. And so on, with 2(q − l) nodes on level l > 0.

(e) Use only (i, j, k, l) for k = i +1 < min(j, l) and i even.

(f) Put ‘1 2q’ first, then ‘2 2q−1’, ... , then ‘q q+1’, then the others. When we branch on ‘0 k’ at the root, for 1 ≤ k ≤ 2q, no options remain for item 2q +1 − k.

(g) ‘0 k’ and ‘2q+1−k l’ are excluded, for all l ∉ {0,k, 2q+1 − k}. (Altogether (2q)(2q − 2) cases.) [Is it perhaps feasible to order the options dynamically?]

216. The search tree is almost always smaller than that of answer 215(c), which in fact has the worst case on every level. But it rarely seems to go below half of the worst-case size. (The author discovered the trick of answer 215(f) by studying randomly generated examples that had unusually small trees.)

Algorithm X needs 540 Gμ to prove that K_21 has no perfect matching. It has potentially 2(214)=11970 excludable quadruples. We can use Algorithm 3.4.2S to sample just m of them; then the running time for m = (2000, 4000, 6000, 8000, and 10000) decreases to about (40 Gμ, 1.6 Gμ, 145 Mμ, 31 Mμ, 12 Mμ), respectively.

217. Each delta α − α′ has k positive terms and k negative terms; we can assume that 1 ≤ k ≤ 4. Furthermore it suffices to work with “normalized” deltas, which are lexicographically smallest under rotation, reflection, and negation. The pentominoes (O, P, . . . , Z) have (10, 64, 81, 73, 78, 25, 23, 24, 22, 3, 78, 24) normalized deltas, of which (1, 7, 3, 3, 2, 0, 1, 0, 1, 0, 4, 0) have k = 1. Two of the deltas are shared by four different pentominoes: 00+01−23−33 (Q, S, W, Z); 00−02 (P, Q, R, Y). Eleven are shared by three.

A common delta is necessary but not sufficient; if α − α′ = β′ − β, we still need to fill in cancelled terms that don’t clash. For example, 00 − 23 is common to Q and W, but it doesn’t yield a bipair. Furthermore (although the exercise didn’t state this!), we don’t want the 10-cell region to have a hole; the delta 00 + 01 − 12 − 22 is common to P, U, and Y, but only PY makes a useful bipair. A delta can arise in more than one way: From 00 + 01 + 02 + 03 − 20 − 21 − 22 − 23 we can make a Q with either 10 or 13, and a Y with either 11 or 12; symmetry (and hole removal) yields only one bipair, not four.

The complete catalog has 34 essentially distinct entries. Eighteen of them

have 10-cell shapes with left-right symmetry. Fourteen have transposition symmetry:

The other two are especially interesting because they are asymmetric:

(These two each lead to eight varieties when rotated and reflected, not just four. See J. C. P. Miller in Eureka: The Archimedeans’ Journal 23 (1960), 14–15.)

218. If the only options involving p are ‘p i:0’ and ‘p i:1’, we can’t eliminate item i. [But if they all involve, say, i:0, we could eliminate it; Algorithm P doesn’t go that far.]

219. If option o contains i, but neither p nor q, it can be in a solution only with two other options {o′, o″} that contain {p, q}. But o′ and o″ must then both contain j. [This argument is like the “naked pairs” of sudoku lore. It’s tempting to go further, by also eliminating items i and j; but that could increase the number of solutions.]

220. Let the option be ‘i_1 i_2[:c_2] ... it[:ct]’. We’ve already covered item i = i_1, which is represented by node x. Nodes x+1, x+2, ... represent the other items, possibly with spacers that were inserted when this option was shortened (see exercise 222). We want to commit i_2, ... , it, and to determine whether this causes LEN(p) to become 0 for some primary p ∉ {i_2,...,it}. The tricky part is to be sure that p ∉ {i_2,...,it}; to accomplish this, we set COLOR(ij ) ← x for 1 < j ≤ t. [In detail: Set p ← x + 1; while p > x, set j ← TOP(p), and if j ≤ 0 set p ← ULINK(p), otherwise set COLOR(j) ← x, p ← p+1.]

Then we make a second pass over the option: Set p ← x+1. While p > x, set j ← TOP(p), and if j ≤ 0 set p ← ULINK(p), otherwise commit′ (p, j) and set p ← p+1. Here commit′ (p, j) emulates (54): Set c ← COLOR(p), q ← DLINK(j); while q ≠ j, hide′″ (q) unless COLOR(q) = c > 0, and set q ← DLINK(q). And hide′″ (p) is just like hide(p), but it detects blocking if LEN(y) becomes 0 for some y ≤ N_1 with COLOR(y) ≠ x.

Finally, a third pass undoes our changes: Set p ← x − 1. While p ≠ x, set j ← TOP(p), and if j ≤ 0 set p ← DLINK(p), otherwise uncommit′(p, j) and set p ← p − 1. Here uncommit′(p, j) undoes commit′(p, j) in the obvious way.

It is possible to switch immediately from committing to uncommitting as soon as blocking is detected, by jumping into the middle of a loop (see answer 122).

221. While S > 0, set x ← S, S ← TOP(x), TOP(x) ← i, and do the following: Set q ← x; while q ≥ x, set j ← TOP(q), and if j ≤ 0 set q ← ULINK(q); otherwise if j ≤ N_1 and LEN(j) = 1, go to P9; otherwise set u ← ULINK(q), d ← DLINK(q), ULINK(d) ← u, DLINK(u) ← d, LEN(j) ← LEN(j) − 1, q ← q +1.

222. Set p ← DLINK(i), and do the following steps while p ≠ i: Set p′ ← DLINK(p), q ← p + 1. While q ≠ p, set j ← TOP(q), and if j ≤ 0 set q ← ULINK(q); otherwise if j = S, exit this loop; otherwise set q ← q + 1. Then if q ≠ p, set ULINK(p) ← p +1, DLINK(p) ← p − 1, TOP(p) ← 0 (thereby making a spacer); otherwise set q ← p + 1 and perform the loop in answer 221 while q = p (instead of while q ≥ x). Finally set p ← p′.

223. In accordance with the conventions of exercise 8, we first declare the items of the reduced problem: For 1 ≤ i ≤ N, output the distinguishing mark for secondary items, if i = N_1 + 1; and output the name of item i, if LEN(i) > 0 or i = N = 1. Then we output the remaining options: For 1 ≤ i ≤ N, if LEN(i) > 0, set p ← DLINK(i) and do the following while p ≠ i: Set q ← p − 1 and while DLINK(q) = q − 1 set q ← q − 1. If TOP(q) ≤ 0 (hence i was the leftmost item to survive, in the option following the spacer node q), output the option as explained below. Then set p ← DLINK(p) and repeat.

To output the (possibly shortened) option that follows node q, set q ← q + 1; then, while TOP(q) ≥ 0, output the name of item TOP(q) if TOP(q) > 0, followed by :c if COLOR(q) = c > 0, and set q ← q + 1. (Afterwards, −TOP(q) is the number of the corresponding option in the original input.)

224. Use 3n−3 items p_1, x_1, i_1, ... , pn−1, xn−1, in−1 (in that order), with the options ‘in−k pk xk’, ‘in−k pk xk+1’, ‘in−k xk’, ‘in−k pk+1’, for 1 ≤ k < n − 1, and also ‘i_1 pn−1 xn−1’, ‘i_1 xn−1’. During round k, for 1 ≤ k < n, item in−k is forced by pk.

225. Some options, like ‘Z 01 02 11 20 21’ and ‘U 30 31 41 50 51’, are obviously useless because they cut off a region of fewer than five cells. More of these options are discarded in the larger problem — but only because of piece U. Eight options, like ‘O 10 11 12 13 14’, are useless because they block a corner cell.

The smaller problem also has numerous options like ‘P 02 12 13 22 23’, which turn out to be useless because they block piece X. (That piece has been confined to just eight placements, in order to break symmetry. It has more freedom in the larger problem, and can’t be blocked there.) Round 2 also discovers that options like ‘O 22 23 24 25 26’ would block X, since round 1 has disabled one of X’s eight choices.

226. Since ∑1′=∑k=12n(2n+1−k)αk, it’s clear that ∑1 +∑1′=(2n+1)∑k=12n⁢ αk⁢ = (2n+1) (n+1)n. Similarly S+S′=(2n+1)∑k=12n αk2=(2n+1)2(n+1)n/3.

The relation ∑2′−(2n+1)∑1′=∑2−(2n+1)∑1 holds for any sequence a_1 ...a_2n.

227. (a) $(ij^2 + ik^2). (b) $(i^2j + i^2k). [$(C − ij^2 − ik^2), for large C, will maximize Σ_2.]

228. Well, it certainly surprised the author. Intuitively, we expect small Σ_1 = ∑ kak to be correlated with small Σ_2 = ∑ k^2ak, but not nearly so well. For some mysterious reason, Langford pairings with the same Σ_1 tend to have the same Σ_2, and vice versa!

That’s not always true. For example, 2 8 6 2 3 5 7 4 3 6 8 5 4 1 7 1 and 3 5 7 4 3 8 6 5 4 1 7 1 2 6 8 2 have the same Σ_1 but different Σ_2; 15174895114107638293261110 and 1 4 1 6 7 10 4 5 9 11 6 8 7 5 2 3 10 2 9 3 8 11 have the same Σ_2 but different Σ_1. Yet such exceptions are rare. When n = 7, the four pairings that have Σ_1 = 444 are the same as the four that have Σ_2 = 4440; the six pairings that have the larger value Σ_1 = 448 are the same as the six that have Σ_2 = 4424, which is smaller than 4440. What is going on?

The special nature of Langford pairings does allow us to prove certain curious facts. For example, let jk be the index of the first occurrence of k. The other occurrence is at jk + k + 1; hence ∑k=1njk=(3n−1)n/4. Also ∑k=1njk2=(4n2−1)n/3−12∑1.

229. These pairings can be found by Algorithm 7.2.2L (or its reverse-order variant). But we can also find them via dancing links, using the sharp minimax modification of Algorithm X (or C) in exercise 85: Order options (16) so that ‘i sj sk’ precedes ‘i′ sj′ sk′’ when j′ < j, or when j′ = j and i′ < i (for lex max) or i < i′ (for lex min). Then repeatedly (i) use the minimax algorithm to fill the smallest undetermined slot sj; (ii) move the option that minimally covered sj to the front of the list, and remove all other options that involve sj.

Thus we find 1213248312134101415168961157121013659147151116 in sixteen such steps, all of which are easy (and need less than 110 Kμ) except for the placements of 8 in s_7 (4.5 Mμ) and 12 in s_9 (500 Kμ). The total time (6 Mμ) includes 465 Kμ just for inputting the data in step X1. After placing 8 items, only 12 solutions remain, so it’s slightly faster to switch gears when finishing. (This pairing has Σ_1 = $5240, Σ_2 = $119192, S = $60324; somewhat high but not extreme.)

The lexicographic maximum turns out to be (108) — partially explaining why it is so “remarkable.” It can be obtained in the same fashion, in fewer than 2 Mμ.

230. Assume that all solutions to the exact cover problem contain the same number of options, d. (For example, d = 16 in Fig. 74.) Then we can replace each cost $c by the complementary cost, $(C − c), where C is sufficiently large to make this nonnegative. Solve the problem with the complementary costs; then subtract its total cost from Cd. [It’s convenient to implement a special version of Algorithm X$ that does this automatically, with appropriate changes to the presentation of intermediate and final results.]

231. (a) MAPLEARRAYSMOKETYPES($139);

(b) HAPPYEXILEALLOWPELTS($176);

(c) JAMBSEQUIPTUMORSASSYor     MAGMAEQUIPOUNCEWAKED($197).

Algorithm X$ needs 6 Gμ, 80 Gμ, and 483 Gμ to find these; Algorithm X needs 5 Gμ, 95 Gμ, and 781 Gμ to visit all solutions, of which there are 27, 8017, and 310077. (Section 7.2.2’s trie-based methods are much faster: They need just 12 Mμ, 628 Mμ,13Gμ.)

232. No. Algorithm X$ finds 96 solutions of minimum cost $84; but the true solution in Fig. 74(a) actually costs $86 by this measure. The effects of 16 rounding errors, each potentially changing the result by nearly $1, have invalidated everything. [Therefore the author used $ ⌊2^32d(i, j)⌋ when preparing Fig. 74. This was safe, because the distance between the first 8 solutions and the 9th was greater than 16 — in fact, much greater, although a difference of only 17 would have been convincing.]

233. With costs $ ⌊2^32 ln d(i, j)⌋, we get the same answers (but faster: 1.2+0.2 Gμ).

234. By that measure, every placement of n nonattacking queens (or rooks!) costs

∑k=1n((k−c)2+(pk−c)2)=2∑k=1n(k−c)2=n(n2−1)6,where  c=(n+1)/2.

235. Now the roles are reversed: We’re more interested in the periphery than in the center, and the minimum is easier to compute than the maximum. The minimum cost, $127760, is achievable in four ways, each symmetric; hence we must take K = 17, not K = 9. This computation took only 1.3 Gμ. (The two examples below have different sets of distances, which coincidentally yield the same total cost.) But there’s a unique way to get the maximum cost, $187760, discovered (with K = 9) in 9.7 Gμ:

236. The idea is first to minimize the longest distance; then, placing a queen at that distance in all possible ways, to minimize the next-longest distance; and so on. In other words, if the options are in nondecreasing order by cost, it’s almost like the search for lexicographically minimax solutions, iteratively as in answer 229.

However, there’s a catch: Many options have the same cost. Different orderings of equal-cost options can lead to wildly different lex-min solutions. For example, suppose there are four options, ‘1’ for $1, ‘2’ for $2, ‘1 3’ for $3, and ‘2 3’ for $3. In that order, the minimax solution omits the final option and costs 3N +2N , which is not optimum.

The solution is to add to each option a primary item describing its cost, and to use Algorithm M iteratively by specifying the number of queens of highest costs, keeping this as low as possible until the problem has no solutions. Here are the best such ways to place n queens, for n = 17, 18, and 19:

The author was able to reach n = 47 with dancing-links-based methods, in an afternoon. But he knew that integer programming is significantly faster for “linear” applications such as the n queens problem (see answer 36). So he enlisted the help of Matteo Fischetti; and sure enough, Matteo was able to extend the results dramatically. Here, for example, are optimum placements for n = 32, 64, and 128:

It appears likely that these optimum queen placements have rotational symmetry only when n = 1, 4, 5, 16, and 32. But the solutions for n = 64 and 128 do have 2^6 and 2^12 equivalent mates, because they contain respectively 6 and 12 “tiltable squares” in the sense of exercise 7.2.2–11(c).

(The limiting behavior may not “kick in” until N is quite large. For example, the optimum solution when n = 16 and N = 20 is not the symmetrical one illustrated; the placements 8 11 4 7 5 12 1 16 14 2 15 10 3 13 6 9 have total cost ≈ 2.08 × 10^21, which beats ≈ 2.09 × 10^21. The limiting shape turns out to be optimum if and only if N ≥ 21.)

237. False. For example, the square shown here is the smallest of ≈ 3 billion solutions for which 02 ≡ 20, 03 ≡ 30, 12 ≡ 21, 13 ≡ 31, 42 ≡ 24, 43 ≡ 34.

[1211311003101031036931391]

238. [113307139][211110311193][212111030111393][111211100103331171][111121110004033193171];[997787733][899986997717][989999889777317][989999988579773371][989999998685977773317].

The problems for n = 7 have 1759244 options; yet they were solved in 20 Gμ without preprocessing. Special methods would, however, be required for n ≥ 8.

239. Introduce primary items k and jk, for 1 ≤ k ≤ n and for all j with k ∈ Sj. When Sj = {k_1,...,kt}, there’s an option ‘jk_1 ... jkt’ of cost wj, together with t options ‘ki jki’ of “infinitesimal” cost j for 1 ≤ i ≤ t; also t “slack” options ‘jki’ of cost 0.

For example, suppose the only sets that cover 1 are S_1, S_2, S_3, S_4; and suppose that an optimum set cover uses S_2 and S_4 but neither S_1 nor S_3. Then a maximum-cost solution to this exact cover problem will use option ‘11 ... ’ of cost w_1, ‘31 ... ’ of cost w_3, ‘1 21’ of cost ∊2, and ‘41’ of cost 0 (because the alternative with ‘21’ and ‘1 41’ has smaller additional cost 0 + ⋴4).

[See M. Gondran and M. Minoux, Graphs and Algorithms (1984), exercise 10.35. When finding the k best solutions instead of a single optimum, all solutions that become identical when is set to zero should be counted just once.]

240. Add {WY, CO, NM} and either ID or UT or AZ. Or add {ID, UT, CO, OK}. Or add {SD, MO} and either {IA, OK} or {NE, AR} (a surprise to the author when he posed this problem).

241. No, although it does find the cases where regions of fewer than 6 vertices are cut off. Round 1 discovers that New England can be shrunk to a single item; then Round 2 is able to remove options such as ‘LA AR TN VA MD PA’. Altogether 3983 options and 5 items are removed, at a cost of 8 Gμ.

242. Before visiting a solution in step R2′, use depth-first search to find the connected components of the residual graph. Reject the solution if any such component has a size d for which d < L · ⌈d/(U − 1)⌉.

243. Let W = w_1 + ··· + wn be the sum of all weights. Then we have ∑k=1d(xk−r)2=∑k=1dxk2−2rW+r2d, because ∑k=1dxk=W in an exact cover problem.

244. True: Let G have m edges and n vertices. A solution with k edges between vertices of the same option has total interior cost n(t − 1) − 2k, total exterior cost 2(m − k).

[But answer 246 shows that this can fail with options of different sizes.]

245. For (a), exercise 242 gives 42498 − 25230 = 17268 options of size 7. Minimum cost $58 is discovered in 101 Mμ. For (b), there are 1176310 − 1116759 = 59551 options with population in [43 .. 45] million. In the optimum solution shown below, which was found in 7.7 Gμ, all populations lie in the range [43.51 .. 44.24] million.

246. Minimum exterior cost ($90 and $74, found in 612 and 11 Mμ):

Minimum interior cost ($176 and $230, found in 1700 and 100 Mμ):

247. Use the procedure of answer 8 for raw data entry, but also set COST(j) ← the cost of the current option for p < j ≤ p + k at the beginning of step I5.

Then assign taxes “greedily” by doing the following for k = 1, 2, ... , n: If item k has no options, terminate with an unsolvable problem. Otherwise let c be the minimum cost of k’s options, and set COST(k) ← c; this is the “tax” on k. If c > 0, subtract c from the cost of every option on k’s list; this will affect all nodes of those options.

(The modified costs will be used internally. But all results reported to the user should be expressed in terms of the original costs, by adding the taxes back in.)

After all taxes have been assigned, sort the options by their (new) costs. (The “natural list merge sort,” exercise 5.2.4–12, works well for this purpose, with the COST fields in spacer nodes serving as links.)

Finally, achieve (118) by re-inserting all nodes, in order of cost.

[Taxes could be assessed in many other ways. In general we seek real numbers u_1, ..., un such that cj ≥ ∑ {ui | item i in option j} for 1 ≤ j ≤ m, where u_1 + ··· + un is maximum. This is a linear programming problem, which happens to be dual to the (fractional) exact cover problem of minimizing c_1x_1+···+cmxm such that x_1,...,xm ≥ 0 and ∑ {xj | item i in option j} = 1 for 1 ≤ i ≤ n. An “optimum” taxation scheme, found by a linear programming solver, might make Algorithm C$ significantly faster than it is with the greedy scheme above, even on highly nonlinear XCC problems; careful tests have not yet been made. See M. Gondran and J. L. Laurière, Revue Française d’Automatique, Informatique et Recherche Opérationnelle 8, V-1 (1974), 27–40.]

248. Set t ← ∞, c ← 0, j ← RLINK(0), and do the following while j > 0: Set p ← DLINK(j) and c′ ← COST(p). If p = j or c′ ≥ ϑ, go to C8$. Otherwise set s ← 1, p ← DLINK(p), and loop as follows: If p = j or COST(p) ≥ ϑ, exit the loop; otherwise if s = t, set s ← s + 1 and exit; otherwise if s ≥ L, set s ← LEN(j) and exit; otherwise set s ← s +1, p ← DLINK(p), and continue. After exiting the loop, if s < t or (s = t and c < c′), set t ← s, i ← j, and c ← c′. Finally set j ← RLINK(j).

[The author uses L = 10. He considered doing a complete search, thereby avoiding the frequent updates to LEN in (13), (15), etc.; but that turned out to be a bad idea.]

249. After we’ve seen t costs, we know only that the remaining dk − t are nonnegative. The following algorithm sorts incoming costs into the rightmost positions of a buffer b_0b_1 ...bdk−1, maintaining the best possible lower bound l: Set l ← t ← 0. When seeing a new cost c, set p ← t, y ← 0, r ← 1, and do this while rp > 0: Set x ← bdk−p. If c ≤ x, set r ← 0. Otherwise if p mod k = 0, set l ← l + x − y; set y ← bdk−p−1 ← x, p ← p − 1. After rp = 0, set bdk−p−1 ← c, t ← t +1; if p mod k = 0, set l ← l + c − y. Stop if l ≥ θ.

250. Keep a separate “accumulator” for each character in Z, and another for z if it is present. Look at each active item i: If NAME(i) begins with a character of Z, add COST(DLINK(i)) to the appropriate accumulator. Otherwise if z = 1, add that cost to the accumulator for z. Otherwise if z > 1, use exercise 249 to accumulate costs that are separated by z. If any of the accumulators becomes ≥ T − Cl, go to C8$.

(When Z or z hints are given, step C1$ should verify that they are legitimate.)

251. When all items have been covered, step Z2 will see the signature S[0] = 0, which was initialized in step Z1; Z[0] = 1 is the “success” node ‘⏉’.

252. Notice that this free ZDD is not ordered, because ‘02 12’ appears above ‘20 21’ in the left branch but below ‘20 21’ in the right branch. See exercise 264.

253. Introduce a global variable COUNT; also auxiliary variables c_0c_1 ... indexed by the current level l; also integer variables C[t] indexed by cache location t. Set COUNT ← 0 and C[0] ← 1 in step Z1. If a cache hit occurs in Z2, set COUNT ← COUNT + C[t]; otherwise set cl ← COUNT. Set C[ml] ← COUNT − cl in step Z7.

254. (a) If the options include d different colors for item i, a subproblem has d + 2 distinct cases: Either item i does not appear in any remaining options, or its list has not been purified, or its list has been purified to a particular color. So we reserve ⌈lg(d + 2)⌉ bits for i in the signature. If, for example, d = 4, those three bits will contain one of the codes 000, 001, 010, 011, 100, 101.

[In order to recognize the relevant case, Algorithm Z’s version of the ‘purify’ operation in (55) should set COLOR(i) ← c in the header node for i; the ‘unpurify’ in (57) should set COLOR(i) ← 0; and step Z1 should set COLOR(i) ← 0. That initialization step should also remap i’s colors so that they appear internally as 1, 2, ... , d.]

(b) In large problems σ will occupy several octabytes. Give each item i a new field SIG(i), which is an index to a code table, and a new field WD(i). If LEN(i) ≠ 0, item i will contribute CODE[SIG(i) + COLOR(i)] to octabyte WD(i) of σ.

[If hashing is used for the cache lookup in step Z2, the CODE table can also contain random bits, for convenience in computing a good hash function.]

(c) Operation hide′(p) doesn’t remove node q from list TOP(q), if that list has been purified. But if TOP(q) is included in the signature, we’ll never get a cache hit for solutions with different colors, even when subproblems don’t actually depend on those colors. Therefore we need to know when a secondary item has no active options in its list.

(d) The trick is to decrease LEN(i), while still retaining the nodes on list i. If LEN(i) becomes zero, when i is a secondary item, we can then remove it from the list of active secondary items (whose head is N + 1, by answer 8).

[We can also use this trick in the ‘hide’ routine: Let hide′″ (p) be like hide(p) except that DLINK(u) and ULINK(d) remain unmodified when COLOR(q) < 0; LEN(x) is decreased as usual.] Of course unpurify and unhide′″ should undo purify and hide′″.

Some delicate maneuvers are needed to avoid deactivating a secondary item twice, and to reactivate it at precisely the right time when unpurifying. (The author’s implementation temporarily sets the LEN to −1.)

255. Let Vn=∑k=0n(n−1−2k)(n−kk),  Wn=∑k=0n(n−12k)+(n−1−2k)2)(n−kk). Using the fact that ∑k=0n(kr)(n−kr)=[zn]z2r/(1−z−z2)r+1, we obtain the closed forms Vn = ((n−5)Fn+1+2(n+1)Fn)/5 and Wn = ((5n^2+7n+25)Fn+1−6(n+1)Fn)/25. (See the derivation of 1.2.8–(17).) When N is even, Algorithm Z performs WN − 1 updates and outputs a ZDD with VN + 2 nodes. When N is odd, it performs WN updates and outputs the trivial ZDD ‘⏊’.

256. Let T (N), Z(N), and C(N) be the time, ZDD size, and cache size needed for KN . With (89) the algorithm first spends T (2q)+ T (2r) time to create a ZDD of size Z(2q)+ Z(2r). Then it spends min(T (2q +1),T (2r − 1)) time to learn that no more ZDD nodes are desirable. The cache size is C(2q)+ C(2r) + min(C(2q +1),C(2r − 1)).

257. (a) There are 2n−1+1 signatures: 11 ... 1 and all n-bit strings beginning with 0.

(b) Each nonzero signature σ has 2νσ−1 branches.

258. See (84). Now it’s Vn=vn+∑k=1n−1(n−1k) vk=((72n−342)5n+(375n−875)4n+600⋅3n+1800n2n+1550)/3600. [For example, V_16 = 40454337297; ϖ_16 = 10480142147.]

259. (a) The signatures at level l are {Xl+1,...,Xn} together with all (nl) l-element subsets of {Y_1,...,Yn}. So there are 2n of them; also 2+∑l=0n(n−1)(nl)=n⁢ 2n−1+2 ZDD nodes; and ((n^2 +3n + 4)2n − 4)/4 updates.

(b) Now the signatures are {Xl+1, . . . ,Xn} plus l-element subsets of {Y_1,...,Yl+1}. So we get (n2)+1 cache memos; n^2 + 2 ZDD nodes; (2n^3 + 15n^2 + n)/6 updates.

260. The ménage problem, with ≈ n!/e^2 solutions, leads to unexpected running times: We seem to get roughly order n3/2ρn updates, where ρ ≈ 3.1; but better results are obtained for n ≥ 13 when the MRV heuristic is not used in step Z3! Then the running time may well be Θ(nen), although the ZDD size apparently grows as nρn with ρ ≈ 2.56.

The other problem, with Ln + 2 solutions, needs just 6n + 9 memos, 8n − 9 ZDD nodes, and 34n − 58 updates.

261. (a) Introduce primary items v− and v+ for each vertex v, representing the possibility of passing through v; but omit v− for v ∈ S, and v+ for v ∈ T . Also introduce secondary items v, whose color (if nonzero) represents the path number. The main options are ‘u+ v− u:k v:k’, for each arc u → v and for 1 ≤ k ≤ m. There also are options ‘v− v:0’ for all v /∈ S, and ‘v+ v:0’ for all v ∉ T.

Moreover, we need a way to number each path canonically, so that we don’t get m! equivalent solutions. (The method of exercise 122 does not work with Algorithm Z.) If S = {s_1,...,sp}, introduce primary items xk and secondary items yk for 1 ≤ k ≤ p, with the following options: ‘xk sk:0 yk−1:j yk:j’ and ‘xk sk:(j+1) yk−1:j yk:(j+1)’, for 1 ≤ k ≤ p and 0 ≤ j < k. [Omit the item yk−1:j when k = 1; omit options with yp ≠ m.]

Many of these options can never be used. Algorithm P readily removes them.

(b) Remove unreachable vertices and unreachable arcs from G, if necessary, so that the only sources and sinks are S = {s_1,...,sm} and T = {t_1,...,tm}. Then use items v−, v+, v and the main options of the construction in part (a); but omit any option that specifies sj:k or tj:k for j ≠ k.

(c) This is a trick question, because each path contains exactly one vertex on the diagonal. The problem therefore factors neatly into two independent subproblems. It suffices to find n − 1 vertex-disjoint paths from S = {(0, 1),..., (0,n−1), (1,n),..., (n−1,n)} to T = {(1, 1),..., (n−1,n−1)} in the digraph with vertices (i, j) for 0 ≤ i ≤ j ≤ n, (i, j) ∉ {(0, 0), (0,n), (n, n)}, and arcs (i, j) →(i +1,j), (i, j) →(i, j − 1).

If this problem has Pn solutions, given by a ZDD Z with Mn nodes, the original problem has pn2 solutions, given by a ZDD Z″ with 2Mn nodes. We obtain Z″ by replacing ⏊ in Z with the root of Z′, where Z′ specifies the reflections of the paths of Z.

Algorithm Z needs just 7 gigamems to find P_16 = 992340657705109416 and M_16 = 3803972. (In fact, Pn is known to be ∏1≤i≤j≤k≤n (i+j+k−1)/(i+j+k−2), the number of plane partitions that are totally symmetric: N. Beluhov [to appear] has found a nice way to glue six triangular diagrams together, in kaleidoscope fashion, which establishes a one-to-one correspondence linking these paths to symmetrical diamond tilings like those of exercise 262(b).)

(d) There are exactly 47356 solutions. Algorithm C finds them in 278 Gμ, without preprocessing; but it needs only 760 Mμ, after Algorithm P has removed redundant options. Algorithm Z, by contrast, handles the problem in 92 Gμ, using 7 gigabytes of memo-cache memory (without preprocessing); 940 Mμ and 90 megabytes (with). Hence Algorithm Z is undesirable for problem (d), but essential for problem (c).

262. (a) The ordering of the primary items — the cells of Sn — is critical: Rowwise ordering (left-to-right, top-to-bottom) causes exponential growth; but columnwise ordering (top-to-bottom, left-to-right) yields linear ZDD size, and Θ(n^2) running time.

Furthermore, it turns out to be better not to use the MRV heuristic, when n ≥ 18. Then the number of ZDD nodes is 154440n − 2655855 for all n ≥ 30. Only 2.2 Gμ are needed for n = 32. There are 68719476736=(2)72 solutions for S_16, via exercise 7.1.4–208; for S_32 there are 152326556015596771390830202722034115329 ≈ 1.552^200.

(An Aztec diamond of order m has exactly 2m(m+1)/2 domino tilings; moreover, as m → ∞, the dominoes at the corners are q.s. aligned, except within an “arctic circle” of radius m/2. See W. Jockusch, J. Propp, and P. Shor, arXiv:math/9801068 [math.CO] (1995), 44 pages; H. Cohn, N. Elkies, and J. Propp, Duke Math. J. 85 (1996), 117–166. See also D. Grensing, I. Carlsen, and H.-Chr. Zapp, Philos. Mag. A41 (1980), 777–781.)

[Tilings of the more general shapes Smn considered here, where we replace 16 by 2m and 7 by m−1, are more mysterious. M. Ciucu observes that R(2m)(n−2m) ⊆ Smn ⊆ R(2m)(n+2m), where Rkn is a k × n rectangle; furthermore both R(2m)(n+2m) \ Smn and Smn \ R(2m)(n−2m) are tilable. Richard Stanley has shown, in Discrete Applied Math. 12 (1985), 81–87, that R(2m)n has ~ α2m⁢  μ2mn+1 tilings, for fixed m as n → ∞, where
ak=(1+2)k+1−(1−2)k+122,       μ2m=rj=1m(cos⁢ θj+1+cos2θj),      θj=jπ2m+1.

Hence Smn has Θ(μ2mn) tilings in that limit. But if m = αn as n → ∞, the limiting “arctic curve” outside which dominoes tend to be frozen remains to be discovered.]

There is, incidentally, a beautiful connection between domino tilings and vertex-disjoint paths, discovered by D. Randall (unpublished):

every vertical domino has either a blue or red path
every horizontal domino has blue and red paths, crossed

(b) In this case the triangle coordinates of answer 124 yield linear growth if we use items (x, y) for 0 ≤ x < n + 8, 0 ≤ y < 16, x + y ≥ 8; (x, y)′ for 0 ≤ x < n + 8, 0 ≤ y < 16, 7 ≤ x + y < n + 15. The options are ‘(x, y) (x′,y′)’, where (x′,y′) = (x, y) −{(0, 0), (0, 1), (1, 0)} and both items exist. Then the ZDD size (without MRV) turns out to be 257400n − 1210061, for all n ≥ 7.

The convex triangular regions that can be tiled with diamonds are precisely those that have equally many Δ and ∇ triangles, namely the generalized hexagons Tlmn with sides (l, m, n, l, m, n) for some l, m, n ≥ 0. These tilings are equivalent to plane partitions that fit in an l × m × n box. In fact you can “see” this equivalence, because the diagrams resemble cubies packed into a corner of the box! (David Klarner made this discovery in the 1970s, but didn’t publish it.) Therefore every tiling of Tn has respectively (1, 2, ... , 8, 7, ... , 1) vertical diamonds in rows (1, 2, ... , 15), hence 64 in all; and these occurrences are nested. For example, the middle diagram corresponds to the reverse plane partition shown here. (See exercise 5.1.4–36, from which it follows that the generalized hexagon Tlmn has exactly ∏lmn=∏i=1l∏j=1m∏k=1n(i+j+k−1)/(i+j+k−2) tilings. In particular, we have Π_888 = 5055160684040254910720; Π88(16) = 2065715788914012182693991725390625.)

00012457
1134569c
12368abc
25789bbc
4578accc
459aaccc
569aaccc
bbbbcccc

[In New York J. of Math. 4 (1998), 137–165, H. Cohn, M. Larsen, and J. Propp studied random tilings of Tlmn when l, m, and n approach infinity with constant scaling, and conjectured that they are q.s. “frozen” outside of the largest enclosed ellipse. See also the more general results of C. Boutillier, Annals of Probability 37 (2009), 107–142.]

263. parameters solutions items options Alg C time, space Alg Z time, space ZDD

(a)

organ-pipe order

14772512

32 + 58

256

40 Gμ

23 KB

55 Gμ

4.1 GB

56M

(b)

6 × 10

2339

72 + 0

2032

4.1 Gμ

230 KB

3.1 Gμ

23 MB

11K

(b)

8 × 8, square

16146

77 + 1

2327

20 Gμ

264 KB

14 Gμ

101 MB

59K

(b)

8 × 8, straight

24600

77 + 1

2358

36 Gμ

267 KB

26 Gμ

177 MB

93K

(b)

8 × 8, skew

23619

77 + 1

2446

28 Gμ

275 KB

20 Gμ

137 MB

84K

(b)

8 × 8, ell

60608

77 + 1

2614

68 Gμ

291 KB

44 Gμ

276 MB

183K

(b)

8 × 8, tee

25943

77 + 1

2446

35 Gμ

275 KB

25 Gμ

166 MB

92K

(c)

aaa placed

987816

49 + 42

1514

25 Gμ

149 KB

18 Gμ

646 MB

2.2M

(d)

(7, 0, 3)

137216

64 + 128

3970

8.5 Gμ

642 KB

1.7 Gμ

20 MB

210K

(d)

(7, 3, 4)

41280

70 + 140

4762

3.2 Gμ

769 KB

1.0 Gμ

13 MB

122K

(e)

p=6, WORDS(1200)

1

12 + 1230

14400

17 Gμ

2 MB

25 Gμ

91 MB

14

(f)

kill symmetry

44*

12 + 36

1188

1.3 Gμ

110 KB

0.9 Gμ

10 MB

186

(g)

unmodified

18

1165 + 66

4889

202 Gμ

509 MB

234 Gμ

8.9 GB

2049

(g)

modified

1

1187 + 66

5143

380 Gμ

537 MB

424 Gμ

15 GB

336

(g)

preprocessed

18

446 + 66

666

223 Mμ

66 KB

1.8 Mμ

136 KB

574

* includes solutions that touch all cells

264. Let the primary items be linearly ordered, and let r(o) be the smallest primary item in option o. If (o¯?l:h) is a ZDD node, every option o′ in the subZDD rooted at h has r(o′) > r(o), because o covers r(o) and smaller items have already been covered. Moreover, if l ≠ 0, the option o′ in node l has r(o′) = r(o); and o′ precedes o in the input.

Thus, if we use a stable sorting algorithm to sort the options by decreasing r(o), the ZDD will respect the reverse of this ordering. [This result was proved by Nishino, Yasuda, Minato, and Nagata in their original paper. Unfortunately, the algorithm is usually too slow without MRV, except in special situations like those of exercise 262.]

265. Every solution below any given ZDD node covers the same primary items. If all items are primary, no two visible nodes have the same signature. And the nodes of the chain below every visible node are distinct, because they branch on different options.

Now suppose we have three primary items {p, q, r}, and one secondary item s, with options ‘p’, ‘p r’, ‘p s’, ‘q r’, ‘q s’. If we don’t use MRV, we’ll branch on p. Choice 1, ‘p’, leads to a subproblem with signature 0111 that outputs I2=(q⁢ r¯?0:1), I3=(p⁢¯?0:2). Choice 2, ‘p r’, leads to a subproblem with signature 0101 that outputs I4=(q⁢  s⁢¯?0:1), I5=(p⁢  r⁢¯?3:4). Choice 3, ‘p s’, leads to a subproblem with signature 0110 that outputs I6=(q⁢  r⁢¯?0:1), I7=(p⁢  s⁢¯?5:6). And I_6 = I_2.

A similar example, with items {q_1,q_2,q_3,r_1,r_2,r_3} in place of {q, r}, and with 23 options ‘p’, ‘p ri’, ‘p s’, ‘qi qj’, ‘ri rj’, ‘qi ri’, ‘qi rj’, ‘qj ri’, ‘qi s’, for 1 ≤ i < j ≤ 3, fails when MRV dictates the choices.

266. Let the given shape be specified as a set of integer pairs (x, y). These pairs might simply be listed one by one in the input; but it’s much more convenient to accept a more compact specification. For example, the utility program with which the author prepared the examples of this book was designed to accept UNIX-like specifications such as ‘[14-7]2 5[0-3]’ for the eight pairs {(1, 2), (4, 2), (5, 2), (6, 2), (7, 2), (5, 0), (5, 1), (5, 3)}. (Notice that a pair is included only once, if it’s specified more than once.) The range 0 ≤ x, y < 62 has proved to be sufficient in almost all instances, with such integers encoded as single “extended hexadecimal digits” 0, 1, ... , 9, a, b, ... , z, A, B, ... , Z. The specification ‘[1-3][1-k]’ is one way to define a 3 × 20 rectangle.

Similarly, each of the given polyominoes is specified by stating its piece name and a set T of typical positions that it might occupy. Such positions (x, y) are specified using the same conventions that were used for the shape; they needn’t lie within that shape.

The program computes base placements by rotating and/or reflecting the elements of that set T . The first base placement is the shifted set T_0 = T − (xmin,ymin), whose coordinates are nonnegative and as small as possible. Then it repeatedly applies an elementary transformation, either (x, y) ↦ (y, xmax − x) or (x, y) ↦ (y, x), to every existing base placement, until no further placements arise. (That process becomes easy when each base placement is represented as a sorted list of packed integers (x≪16)+y.) For example, the typical positions of the straight tromino might be specified as ‘1[1-3]’; it will have two base placements, {(0, 0), (0, 1), (0, 2)} and {(0, 0), (1, 0), (2, 0)}.

After digesting the input specifications, the program defines the items of the exact cover problem, which are (i) the piece names; (ii) the cells xy of the given shape.

Finally, it defines the options: For each piece p and for each base placement T′ of p, and for each offset (δx,δy) such that T′ +(δx, δy) lies fully within the given shape, there’s an option that names the items {p}∪{(x + δx,y + δy) | (x, y) ∈ T′}.

(The output of this program is often edited by hand, to take account of special circumstances. For example, some items may change from primary to secondary; some options may be eliminated in order to break symmetry. The author’s implementation also allows the specification of secondary items with color controls, along with base placements that include such controls.)

Historical notes: Early algorithms for polyomino packing failed to realize the essentially unity between cells to be covered and pieces to be covered; their treatment of cells was quite different from their treatment of pieces. The fact that both cells and pieces are primary items of a “pure” exact cover problem was first noticed in connection with the Soma cube, by C. Peter-Orth [Discrete Mathematics 57 (1985), 105–121]. The base placements of tiles that are to be translated (but not rotated or reflected) are called “aspects” in Tilings and Patterns by Grünbaum and Shephard (1987).

267. RUSTY. [Leigh Mercer posed a similar question to Martin Gardner in 1960.]

268. As in the 3 × 20 example considered in the text, we can set up an exact cover problem with 12 + 60 items, and with options for every potential placement of each piece. This gives respectively (52, 292, 232, 240, 232, 120, 146, 120, 120, 30, 232, 120) options for pieces (O, P, ... , Z) in Conway’s nomenclature, thus 1936 options in all.

To reduce symmetry, we can insist that the X occurs in the upper left corner; then it contributes just 10 options instead of 30. But some solutions are still counted twice, when X is centered in the middle row. To prevent this we can add a secondary item ‘s’: Append ‘s’ to the five options that correspond to those centered appearances; also append ‘s’ to the 60 options that correspond to placements where the Z is flipped over.

Without those changes, Algorithm X would use 10.04 Gμ to find 4040 solutions; with them, it needs just 2.93 Gμ to find 1010.

This approach to symmetry breaking in pentomino problems is due to Dana Scott [Technical Report No. 1 (Princeton University Dept. of Electrical Engineering, 10 June 1958)]. Another way to break symmetry would be to allow X anywhere, but to restrict the W to its 30 unrotated placements. That works almost as well: 2.96 Gμ.

269. There’s a unique way to pack P, Q, R, U, X into a 5 × 5 square, and to pack the other seven into a 5 × 7. (See below.) With independent reflections, together with rotation of the square, we obtain 16 of the 1010. There’s also a unique way to pack P, R, U into a 5 × 3 and the others into a 5 × 9 (noticed by R. A. Fairbairn in 1967), yielding 8 more. And there’s a unique way to pack O, Q, T, W, Y, Z into a 5 × 6, plus two ways to pack the others via a bipair, yielding another 16. (These paired 5 × 6 patterns were apparently first noticed by J. Pestieau; see answer 286.) Finally, the packings in the next exercise give us 264 decomposable 5 × 12s altogether.

[Similarly, C. J. Bouwkamp discovered that S, V, T, Y pack uniquely into a 4 × 5, while the other eight can be put into a 4 × 10 in five ways, thus accounting for 40 of the 368 distinct 4 × 15s. See JRM 3 (1970), 125.]

270. Without symmetry reduction, 448 solutions are found in 1.24 Gμ. But we can restrict X to the upper left corner, as in answer 268, flagging its placements with ‘s’ when centered in the middle row or middle column (but not both). Again the ‘s’ is appended to flipped Z’s. Finally, when X is placed in dead center, we append another secondary item ‘c’, and append ‘c’ to the 90°-rotated placements of W. This yields 112 solutions, after 0.35 Gμ.

Or we could leave X unhindered but curtail W to 1/4 of its placements. That’s easier to do (although not quite as clever) and it finds those 112 in 0.44 Gμ.

Incidentally, there aren’t actually any solutions with X in dead center.

271. The exact cover problem analogous to that in exercise 268 has 12 + 60 items and (56, 304, 248, 256, 248, 128, 152, 128, 128, 32, 248, 128) options. It finds 9356 solutions after 16.42 Gμ of computation, without symmetry reduction. But if we insist that X be centered in the upper left quarter, by removing all but 8 of its placements, we get 2339 solutions after just 4.11 Gμ. (The alternative of restricting W’s rotations is not as effective in this case: 5.56 Gμ.) These solutions were first enumerated by C. B. and Jenifer Haselgrove [Eureka: The Archimedeans’ Journal 23 (1960), 16–18].

272. (a) Obviously only k = 5 is feasible. All such packings can be obtained by omitting all options of the cover problem that straddle the “cut.” That leaves 1507 of the original 2032 options, and yields 16 solutions after 104 Mμ. (Those 16 boil down to just the two 5 × 6 decompositions that we already saw in answer 269.)

(b) Now we remove the 763 options for placements that don’t touch the boundary, and obtain just the two solutions below, after 100 Mμ. (This result was first noticed by Tony Potts, who posted it to Martin Gardner on 9 February 1960.)

(c) With 1237 placements/options, the unique solution is now found after 83 Mμ.

(d) There are respectively (0, 9, 3, 47, 16, 8, 3, 1, 30, 22, 5, 11) solutions for pentominoes (O, P, Q, ... , Z). (The I/O pentomino can be “framed” by the others in 11 ways; but all of those packings also have at least one other interior pentomino.)

(e) Despite many ways to cover all boundary cells with just seven pentominoes, none of them lead to an overall solution. Thus the minimum is eight; 207 of the 2339 solutions attain it. To find them we might as well generate and examine all 2339.

(f) The question is ambiguous: If we’re willing to allow the X to touch unnamed pieces at a corner, but not at an edge, there are 25 solutions (8 of which happen to be answers to part (a)). In each of these solutions, X also touches the outer boundary. (The cover and frontispiece of Clarke’s book show a packing in which X doesn’t touch the boundary, but it doesn’t solve this problem: Using Golomb’s piece names, there’s an edge where X meets I, and there’s a point where X meets P.) There also are two packings in which the edges of X touch only F, N, U, and the boundary, but not V.

On the other hand, there are just 6 solutions if we allow only F, N, U, V to touch X’s corner points. One of them, shown below, has X touching the short side and seems to match the quotation best. These 6 solutions can be found in just 47 Mμ, by introducing 60 secondary items as sort of an “upper level” to the board: All placements of X occupy the normal five lower-level cells, plus up to 16 upper-level cells that touch them; all placements of F, N, U, V are unchanged; all placements of the other seven pieces occupy both the lower and the upper level. This nicely forbids them from touching X.

273. (a) We could set this up as twelve separate exact cover problems, one for each pentomino omitted. But it’s more interesting to consider all cases simultaneously, by giving a “free pass” to one pentomino as follows: Add a new primary item ‘#’, and twelve new options ‘# O’, ‘# P’, ... , ‘# Z’. The sixty items ij are demoted to secondary status.

To remove symmetry, delete 3/4 of the options for piece V; also make its new option ‘# V s’, and add ‘s’ to 3/4 of the options for piece W, where ‘s’ is a new secondary item. That makes a total of 1194 options, involving 13 + 61 items.

If Algorithm X branches first on #, the effect is equivalent to 12 separate runs; the search tree has 7.9 billion nodes, and the run time is 16.8 teramems. But if we use the nonsharp preference heuristic (see answer 10), the algorithm is able to save some time by making decisions that are common to several subcases. Its search tree then has 7.3 billion nodes, and the run time is 15.1 teramems. Of course both methods give the same answer, which is huge: 118,034,464.

(b) Now keep items ij primary, but introduce 60 new secondary items ij′. There are 60 new options ‘ij ij′ (i+1)j′ i(j+1)′ (i+1)(j+1)′’, where we omit items containing (i+1) when i = 2 or (j+1) when j = 19. This problem has 1254 options involving 73+61 items. Its search tree (with deprecated # branching) has about 950 million nodes; it finds 4,527,002 solutions, after about 1.5 teramems of computation.

A related, but much simpler, problem asks for packings in which exactly one hole appears in each of the column pairs {1, 2}, {5, 6}, {9, a}, {d, e}, {h, i}. That one has 1224 options, 78+1 items, 20 meganodes, 73 gigamems, and 23642 solutions. Here’s one:

(c) A setup like the one in (a) yields 1127 options, 13+58 items, 1130 meganodes, 2683 gigamems, 22237 solutions. (One of the noteworthy solutions is illustrated above.)

274. Restrict X to five essentially different positions; if X is on the diagonal, also keep Z unflipped by using the secondary item ‘s’ as in answer 268. There are respectively (16146, 24600, 23619, 60608, 25943) solutions, found in (20.3, 36.3, 28.0, 68.3, 35.2) Gμ.

In each case the tetromino can be placed anywhere that doesn’t immediately cut off a region of one or two squares. [The twelve pentominoes first appeared in print when H. E. Dudeney published The Canterbury Puzzles in 1907. His puzzle #74, “The Broken Chessboard,” presented the first solution shown above, with pieces checkered in black and white. That parity restriction, with the further condition that no piece is turned over, would reduce the number of solutions to only 4, findable in 120 Mμ.]

The 60-element subsets of the chessboard that can’t be packed with the pentominoes have been characterized by M. Reid in JRM 26 (1994), 153–154.

The earliest known polyomino puzzle appeared in P. F. Catel’s Verzeichniß von sämmtlichen Waaren (Berlin, 1785), #11: 4 Z pentominoes + 4 ells make a 6×6 square.

275. Yes, in seven essentially different ways. To remove symmetry, we can make the O vertical and put the X in the right half. (The pentominoes will have a total of 6 × 2+5 × 3 + 4 = 31 black squares; therefore the tetromino must be .)

276. These shapes can’t be packed in a rectangle. But we can use the “supertile” to make an infinite strip . [See B. Grünbaum and G. C. Shephard, Tilings and Patterns (1987), 508.] We can also tile the plane with a supertile like , or even use a generalized torus such as (see exercise 7–137). That supertile was used in 2009 by George Sicherman to make tetromino wallpaper.

277. The 2339 solutions contain 563 that satisfy the “tatami” condition: No four pieces meet at any one point. Each of those 563 leads to a simple 12-vertex graph coloring problem; for example, the SAT methods of Section 7.2.2.2 typically need at most two or three kilomems to decide each case.

It turns out that exactly 94 are three-colorable, including the second solution to exercise 272(b). Here are the three for which W, X, Y, Z all have the same color:

278. The 2339 solutions in answer 271 restrict X to the upper left quarter; we must be careful not to include bipairs that might swap X out of that region. One way (see exercise 212) is to order the items: Put X first, then the other piece names, then the place names from 00 to 59. All swaps involving X will then move it up or left.

The 34 bipairs of the catalog now result in an exact cover problem with the same primary items and options as before, but with 2804 new secondary items. They limit the number of solutions to 1523; but the running time increases to 4.26 Gμ.

[The proof idea of Theorem S yields an interesting directed acyclic graph with 2339 vertices and 937 arcs. It has 1528 source vertices, 1523 sink vertices, and 939 isolated vertices (both sources and sinks). If we ignore the arc directions, there are 1499 components, of which the largest has size 10. That component contains the leftmost solution below, which belongs to four different bipairs. There also are two components of size 8, with three nonoverlapping bipairs. The rightmost solution belongs to a component of size 6, which would grow to size 8 if X were allowed to move downward.]

279. It’s also possible to wrap two cubes of size 5×5×5, as shown by F. Hansson; see Fairy Chess Review 6 (1947–1948), problems 7124 and 7591. A full discussion appears in FGbook, pages 685–689.

280. (Notice that width 3 would be impossible, because every faultfree placement of the V needs width 4 or more.) We can set up an exact cover problem for a 4 × 19 rectangle in the usual way; but then we make cell (x, y + 15) identical to (3 − x, y) for 0 ≤ x < 4 and 0 ≤ y < 5, essentially making a half-twist when the pattern begins to wrap around. There are 60 symmetries, and care is needed to remove them properly. The easiest way is to put X into a fixed position, and allow W to rotate at most 90°.

This exact cover problem has 850 solutions, 502 of which are faultfree. Here’s one of the 29 strongly three-colorable ones, shown before and after its ends are joined:

281. Both shapes have 8-fold symmetry, so we can save a factor of nearly 8 by placing the X in (say) the north-northwest octant. If X thereby falls on the diagonal, or in the middle column, we can insist that the Z is not flipped, by introducing a secondary item ‘s’ as in answer 270. Furthermore, if X occurs in dead center — this is possible only for shape (i) — we use ‘c’ as in that answer to prohibit also any rotation of the W.

Thus we find (a) 10 packings, in 3.5 Gμ; (b) 7302 packings, in 353 Gμ; for instance

It turns out that the monomino must appear in or next to a corner, as shown. [The first solution to shape (i) with monomino in the corner was sent to Martin Gardner by H. Hawkins in 1958. The first solution of the other type was published by J. A. Lindon in Recreational Mathematics Magazine #6 (December 1961), 22. Shape (ii) was introduced and solved much earlier, by G. Fuhlendorf in The Problemist: Fairy Chess Supplement 2, 17 and 18 (April and June, 1936), problem 2410.]

282. It’s easy to set up an exact cover problem in which the cells touching the polyomino are primary items, while other cells are secondary, and with options restricted to placements that contain at least one primary item. Postprocessing can then remove spurious solutions that contain holes. Typical answers for (a) are

representing respectively (9, 2153, 37, 2, 17, 28, 18, 10, 9, 2, 4, 1) cases. For (b) they’re

representing (16, 642, 1, 469, 551, 18, 24, 6, 4, 2, 162, 1). The total number of fences is respectively (3120, 1015033, 8660380, 284697, 1623023, 486, 150, 2914, 15707, 2, 456676, 2074), after weeding out respectively (0, 0, 16387236, 398495, 2503512, 665, 600, 11456, 0, 0, 449139, 5379) cases with holes. (See MAA Focus 36, 3 (June/July 2016), 26; 36, 4 (August/September 2016), 33.) Of course we can also make fences for one shape by using other shapes; for example, there’s a beautiful way to fence a Z with 12 Ps, also a unique way to fence one pentomino with only three copies of another.

283. The small fences of answer 282(a) already meet this condition — except for the X, which has no tatami fence. The large fences for T and U in 282(b) are also good. But the other nine fences can no longer be as large:

[The tatami condition can be incorporated into the exact cover problem by introducing a secondary item /ij for each interior point ij. Add this item to every placement option that has a convex corner at ij and occupies either the cell to the northeast or the cell to the southwest. However, for this exercise it’s best simply to apply the tatami condition directly to each ordinary solution, before postprocessing for hole-removal.]

284. This problem is readily solved with the “second death” algorithm of exercise 19, by letting the four designated piece names be the only primary items. The answers to both (a) and (b) are unique. [See M. Gardner, Scientific American 213, 4 (October 1965), 96–102, for Golomb’s conjectures about minimum blocking configurations on larger boards.]

285. This exercise, with 3 × 30, 5 × 18, 6 × 15, and 9 × 10 rectangles, yields four increasingly difficult benchmarks for the exact cover problem, having respectively (46, 686628, 2567183, 10440433) solutions. Symmetry can be broken as in answer 270. The 3×30 case was first resolved by J. Haselgrove; the 9×10 packings were first enumerated by A. Wassermann and P. Östergård, independently. [See New Scientist 12 (1962), 260– 261; J. Meeus, JRM 6 (1973), 215–220; and FGbook pages 455, 468–469.] Algorithm X needs (.006, 5.234, 15.576, 63.386) teramems to find them.

286. Two solutions are now equivalent only when related by 180° rotation. Thus there are 2 · 2339/64 = 73.09375 solutions per problem, on average. The minimum (42) and maximum (136) solution counts occur for the cases

[In U.S. Patent 2900190 (1959, filed 1956), J. Pestieau remarked that these 64 problems would give his pentomino puzzle “unlimited life and utility.”]

287. Let c = (12, 11,..., 1) for pieces (O, P,..., Z) when assigning costs to each option. Algorithm X$, when told that every option contains one piece and five cells, finds

in respectively (1.5, 3.4, 3.3, 2.9, 3.2, 1.4, 1.1) Gμ. The corresponding times for Algorithm X are (3.7, 10.0, 16.4, 16.4, 10.0, 3.7, 2.0) Gμ. (However, we could reduce symmetry when applying Algorithm X, then calculate the values of four or eight different reflections or rotations whenever a solution is found; that would often be faster.)

288. When symmetry is removed efficiently, Algorithm X needs 63 Tμ to visit all of the essentially different solutions. But Algorithm X$ wins this competition, by discovering

(which both are uniquely optimum) in 28.9 Tμ and 25.1 Tμ, respectively.

289. (a) One of the 8·2422·85·263·95·224·262·226·228·96·105·174 solutions is shown in Fig. A–4. (It isn’t hard to keep pentominoes of the same shape from touching.)

(b) Now there are 1472 · 5915 · 596 · 251 · 542 · 204 · 170 · 226 · 228 · 96 · 651 · 316.

(c) The first seven columns left of the middle line can yield six 12-cell regions only by using all 72 cells. Thus the problem factors neatly into ten independent problems of the form (i). That problem has 7712 solutions with six connected regions; Algorithm X$ needs a search tree of only 622 nodes to determine that there are just 11 minimum-perimeter solutions. Three of them are symmetrical; and the nicest is shown in (ii). (And two of the solutions, such as (iii), maximize the total perimeter.)

Unfortunately (36) can’t be expanded into the desired 720-cell shape based on (ii), because the scaled-up Q can’t be packed. But the alternative form of (36) does lead to 16 · 2139 · 6 · 97 · 259 · 111 · 44 · 64 · 79 · 12 · 17 · 111 solutions, such as the one in Fig. A–4.

290. There are no ways to fill 2 × 20; 4 · 66 ways to fill 4 × 10; 4 · 84 ways to fill 5 × 8. None of the solutions are symmetrical. [See R. K. Guy, Nabla 7 (1960), 99–101.]

291. The puzzles for January, April, September, and December (say) are equivalent; thus only 4·31 = 124 puzzles need to be solvable, not 366. Only 53 of the 220 pentomino triples are unsuitable: First reject all 55 that include X, and all 10 that are subsets of {O, R, S, W, Z}; then restore P{O, Q, S, T, U, V, Y}X and ORS, OSW, RSW; then reject RTZ and TWZ. Of the remaining 167 triples, PQV is by far the easiest: Every PQV puzzle has at least 1778 solutions! The hardest is QTX, which allows only about 33 solutions per day, on average. [This puzzle was designed by Marcel Gillen, © 2018, who made it with pentominoes R, U, W for the 2018 International Puzzle Party.]

292. Most of the hexominoes will have three black cells and three white cells, in any “checkering” of the board. However, eleven of them (shown as darker gray in the illustration) will have a two-to-four split. Thus the total number of black cells will always be an even number between 94 and 116, inclusive. But a 210-cell rectangle always contains exactly 105 black cells. [See The Problemist: Fairy Chess Supplement 2, 9–10 (1934–1935), 92, 104–105; Fairy Chess Review 3, 4–5 (1937), problem 2622.]

Fig. A–4. Pentominoes of pentominoes.

Benjamin’s triangular shape, on the other hand, has 1+3+5+···+19 = 10^2 = 100 cells of one parity and (212)−102=110 of the other. It can be packed with the 35 hexominoes in a huge number of ways, probably not feasible to count exactly.

293. The parity considerations in answer 292 tell us that this is possible only for the “unbalanced” hexominoes, such as the one shown. And in fact, Algorithm X readily finds solutions for all eleven of those, too numerous to count. Here’s an example:

[See Fairy Chess Review 6 (April 1947) through 7 (June 1949), problems 7252, 7326, 7388, 7460, 7592, 7728, 7794, 7865, 7940, 7995, 8080. See also the similar problem 7092.]

294. Each castle must contain an odd number of the eleven unbalanced hexominoes (see answer 292). Thus we can begin by finding all sets of seven hexominoes that can be packed into a castle: This amounts to solving (111)+(113)+(115)+(117)=968 exact cover problems, one for each potential choice of unbalanced elements. Each of those problems is fairly easy; the 24 balanced hexominoes provide secondary items, while the castle cells and the chosen unbalanced elements are primary. In this way we obtain 39411 suitable sets of seven hexominoes, with only a moderate amount of computation.

That gives us another exact cover problem, having 35 items and 39411 options. This secondary problem turns out to have exactly 1201 solutions (found in just 115 Gμ), each of which leads to at least one of the desired overall packings. Here’s one:

In this example, two of the hexominoes in the rightmost castle can be flipped vertically; and of course the entire contents of each castle can independently be flipped horizontally. Thus we get 64 packings from this particular partition of the hexominoes (or maybe 64 · 5!, by permuting the castles), but only two of them are “really” distinct. Taking multiplicities into account, there are 1803 “really” distinct packings altogether.

[Frans Hansson found the first way to pack the hexominoes into five equal shapes, using as the container; see Fairy Chess Review 8 (1952–1953), problem 9442. His container admits 123189 suitable sets of seven, and 9298602 partitions into five suitable sets instead of only 1201. Even more packings are possible with the container , which has 202289 suitable sets and 3767481163 partitions!]

In 1965, M. J. Povah packed all of the hexominoes into containers of shape , using seven sets of five; see The Games and Puzzles Journal 2 (1996), 206.

295. By exercise 292, m must be odd, and less than 35. F. Hansson posed this question in Fairy Chess Review 7 (1950), problem 8556. He gave a solution for m = 19,

and claimed without proof that 19 is maximum. The 13 dark gray hexominoes in this diagram cannot be placed in either “arm”; so they must go in the center. (Medium gray indicates pieces that have parity restrictions in the arms.) Thus we cannot have m ≥ 25.

When m = 23, there are 39 ways to place all of the hard hexominoes, such as

However, none of these is completable with the other 22; hence m ≤ 21.

When m = 21, the hard hexominoes can be placed in 791792 ways, without creating a region whose size isn’t a multiple of 6 and without creating more than one region that matches a particular hexomino. Those 791792 ways have 69507 essentially distinct “footprints” of occupied cells, and the vast majority of those footprints appear to be impossible to fill. But in 2016, George Sicherman found the remarkable packing

which not only solves m = 21, it yields solutions for m = 19, 17, 15, 11, 9, 7, 5, and 3 by simple modifications. Sicherman also found separate solutions for m = 13 and m = 1.

296. Stead’s original solution makes a very pleasant three-colored design:

[See Fairy Chess Review 9 (1954), 2–4; also FGbook, pages 659–662.]

This problem is best solved via the techniques of dynamic programming (Section 7.7), not with Algorithm X, because numerous subproblems are equivalent.

297. Yes — in fact, there are so many ways, further conditions ought to be imposed. Torbijn’s original quest, to leave a hexomino-shaped “hole” in one square, turns out to have been impossible. But there’s a nice alternative: We can add the two trominoes.

A. van de Wetering showed in 1991 that exactly 13710 sets of six hexominoes can fit into a single square. [See JRM 23 (1991), 304–305.] Similarly, exactly 34527 sets of five hexominoes will fit, when supplemented by two trominoes that both occupy two black cells. So we’re left with a secondary covering problem, with 35 primary items and 48237 options, as in answer 294. That problem has 163 solutions (found in 3 Tμ).

Another alternative, also suggested by van de Wetering, is to place six empty cells symmetrically. He also was able to add a monomino and one of the pentominoes: The secondary covering problems associated with pentominoes (O, P, ... , Z) turn out to have (94, 475, 1099, 0, 0, 2, 181, 522, 0, 0, 183, 0) solutions.

298. Make options for the pentominoes in cells xy for 0 ≤ x < 8, 0 ≤ y < 10 as in exercise 266, and also for the tetrominoes in cells xy for 1 ≤ x < 7, 1 ≤ y < 9. In the latter options include also items xy′:0 for all cells xy in the tetromino, as well as xy′:1 for all other cells xy touching the tetromino, where the items xy′ for 0 ≤ x < 8 and 0 ≤ y < 10 are secondary. We can also assume that the center of the X pentomino lies in the upper left corner. There are 168 solutions, found after 1.5 Tμ of computation. (Another way to keep the tetrominoes from touching would be to introduce secondary items for the vertices of the grid. Such items are more difficult to implement, however, because they behave differently under the rotations of answer 266.)

[Many problems that involve placing the tetrominoes and pentominoes together in a rectangle were explored by H. D. Benjamin and others in the Fairy Chess Review, beginning already with its predecessor The Problemist: Fairy Chess Supplement 2, 16 (February 1936), problem 2171. But this question seems to be new; it was inspired by Michael Keller’s 15 × 18 pentomino + hexomino construction in World Game Review 9 (1989), 3. See also P. Torbijn’s elegant 13 × 23 packing of all the n-ominoes for 1 ≤ n ≤ 6, in Cubism For Fun 25, part 1 (1990), 11.]

299. P. J. Torbijn and J. Meeus [JRM 32 (2003), 78–79] have exhibited solutions for rectangles of sizes 6 × 45, 9 × 30, 10 × 27, and 15 × 18; thus intuition suggests that enormously many solutions ought to be possible for this case too. But Peter Esser has surprisingly proved that no packing of the 35 hexominoes into a 5 × 54 rectangle will occupy all 114 of the border cells. Indeed, the pieces can individually occupy at most (6, 5, 5, 4, 4, 4, 3, 3, 3, 3, 3, 3, 3, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 5 + x_24, 4 + x_25, 4 + x_26, 4+x_27, 3+x_28, 3+x_29, 3+x_30, 2+x_31, 2+x_32, 4+2x_33, 3+2x_34, 3+2x_35) border cells, respectively, under an appropriate numbering of the pieces, where xk = 1 only if piece k is in a corner. Since there are only four corners, we can occupy at most 6+5+···+4+3+ 3+(1+2+2+2) = 114 border cells — but only if x_33 = x_34 = x_35 = 1. Unfortunately, those last three pieces (namely , , ) can’t simultaneously occupy corners.

300. Make options as usual (exercise 266), but also include 100 new options ‘xy Rx Cy’ for 0 ≤ x, y < 10. Then use Algorithm M, assigning multiplicity 4 to each Rx and Cy. Remove symmetry by confining X to the upper left corner, and by insisting that O be horizontal. (a) One of the 31 solutions (found in 12 Gμ) is shown below. (b) This case has 5347 solutions (found in 4.6 Tμ); and if we insist on filling also all cells just above the diagonals, the solution turns out to be unique (see below). (c) Instead of focusing on diagonals, Aad van de Wetering noticed that we can require the empty spaces to be symmetrical. For example, there are 1094 solutions (found in 19.2 Tμ) whose empty spaces are diagonally symmetric. Three of them, like the one shown here, are also rather close (92%) to being centrally symmetric (that is, under 180° rotation).

Three others, like the fourth example above, leave a 4 × 4 hole in the corner. Moreover, there are 98 solutions (found in 3.2 Tμ) whose empty spaces have 100% central symmetry. One of them has a large “moat” between two blocks of pentominoes; another has connected pentominoes, with holes of size at least 6.

Furthermore, van de Wetering reported that he had found “by accident” a solution where each of the four 5 × 5 quadrants of the 10 × 10 contained exactly three pentominoes. This additional stipulation is, indeed, easy to add to our MCC formulation: We omit options that cross quadrant boundaries, append a new item Qt to each option in the tth quadrant, and give multiplicity 3 to each Qt. It turns out that there actually are 1,124,352 inequivalent solutions(!), found by Algorithm M in 23 Tμ.

But van de Wetering also discovered a class of solutions that’s even more interesting: He packed the empty spaces entirely with “ghost” pentominoes, all different!

To obtain such remarkable solutions, use primary items #xy, !xy, #Rx, and #Cy for 0 ≤ x, y < 10, as well as O, P, ... , Z; use secondary items xy as well as O′, P′, ... , Z′. Items #Rx and #Cy have multiplicity 4. Specify two options for each pentomino placement, such as ‘V !00 00:1 !01 01:1 !02 02:1 !10 10:1 !20 20:1’ for V in the corner and ‘V′ !00 00:0 !01 01:0 !02 02:0 !10 10:0 !20 20:0’ for its ghost in that place. Also specify 200 further options, ‘#xy #Rx #Cy xy:0’ and ‘#xy xy:1’, for 0 ≤ x, y < 10. Algorithm M with the nonsharp heuristic will then make intelligent choices. There are (amazingly) 357 solutions, found in 322 teramems with a search tree of 32 giganodes. The first solution above is one of six that cover exactly six cells of each main diagonal, answering a question that had been posed by Aad Thoen. The second solution is one of two for which all seven of the “unambiguously named pentominoes” T, U, V, W, X, Y, Z are among the ghosts. The third solution is one of two that respects 5×5 quadrants. [Note: A similar question, but with identical polyominoes, was Erich Friedman’s “problem of the month” in May 2007; see erich-friedman.github.io/mathmagic/0507.xhtml.]

301. (a) Algorithm M produces 4 · 13330 solutions when we specify the desired multiplicities for cell items. Symmetry under reflection can be removed by restricting, say, W to only 1/4 of its options.

(b) Consider the conflict graph on vertices O, P, ... , Z, defined by declaring pieces to be adjacent when they appear in the same cell. We can achieve ≤ d levels if and only if we can color that graph with ≤ d colors. The conflict graph for the given arrangement has the 4-clique {Q, X, Y, Z}; so it can’t be 3-colored.

(c, d) A SAT solver such as Algorithm 7.2.2.2D quickly determines that exactly (587, 12550, 193) of the conflict graphs for the 13330 distinct solutions to (a) have chromatic numbers (3, 4, 5). The first example below can be (uniquely) 3-colored OVYZ | PRWX | QSTU; the second example has the clique {Q, R, S, W, Y}.

302. (a) There are 94. (But 16 of them have interior “holes” and can’t be used in (b).)

(b) The two solutions are related by rotating four of the pieces:

(c) Sixteen different jigsaw sudoku diagrams can be used. The first of them collaborates with π as shown above; the others probably do too. [Appendix E has the answer. This exercise was suggested by E. Timmermans, Cubism For Fun 85 (2011), 4–9.]

303. (a) Represent the tree as a sequence a_0a_1 ... a2n−1 of nested parentheses; then a_0 will match a2n−1. The left boundary of the corresponding parallomino is obtained by mapping each ‘(’ into N or E, according as it is immediately followed by ‘(’ or ‘)’. The right boundary, similarly, maps each ‘)’ into N or E according as it is immediately preceded by ‘)’ or ‘(’. For example, if we take 7.2.1.6–(1) and enclose it in an additional pair of parentheses, the corresponding parallomino is shown below with part (d).

(b) This series wxy + w^2(xy^2 + x^2y) + w^3(xy^3 + 2x^2y^2 + x^3y) + ··· can be written wxyH(w, wx, wy), where H(w, x, y) = 1/(1 − x − y − G(w, x, y)) generates a sequence of “atoms” corresponding to places x, y, G where the juxtaposed boundary paths have the respective forms EE, NN, or NE〈inner〉EN. The area is thereby computed by diagonals between corresponding boundary points. (In the example from (a), the area is 1+1+1+1+2+2+2+2+2+2+2+2+2+1+1; there’s an “outer” G, whose H is yxyGy, and an “inner” G, whose H is xyyxyxxy.) Thus we can write G as a continued fraction,

G(w,x,y)=wxy/(1−wx−wy−w3xy/(1−w2x−w2y/(…))).

[A completely different form is also possible, namely G(w,x,y)=xJ1(w,x,y)J0(w,x,y), where

J0(w,x,y)=Σn=0∞(−1)nynwn(n+1)/2(1−w)(1−w2)…(1−wn)(1−xw)(1−xw2)…(1−xwn);J1(w,x,y)=Σn=1∞(−1)n−1ynwn(n+1)/2(1−w)(1−w2)…(1−wn−1)(1−xw)(1−xw2)…(1−xwn).

This form, derived via horizontal slices, disguises the symmetry between x and y.]

(c) Let G(w, z) = G(w, z, z). We want [zn] G′ (1, z), where differentiation is with respect to the first parameter. From the formulas in (b) we know that G(1, z) = z(C(z) − 1), where C(z)=(1−1−4z)/(2z) generates the Catalan numbers. Partial derivatives ∂/∂w and ∂/∂z then give G′(1, z) = z^2/(1−4z) and G′(1,z)=1/1−4z−1.

(d) This problem has four symmetries, because we can reflect about either diagonal. When n = 5, Algorithm X finds 801 × 4 solutions, of which 129 × 4 satisfy the tatami condition, and 16 × 4 are strongly three-colorable. (The tatami condition is easily enforced via secondary items in this case, because we need only stipulate that the upper right corner of one parallomino doesn’t match the lower left corner of another.) When n = 6 there are oodles and oodles of solutions. All of the trees/parallominoes thereby appear together in an attractive compact pattern.

[References: J. Levine, Scripta Mathematica 24 (1959), 335–338; D. A. Klarner and R. L. Rivest, Discrete Math. 8 (1974), 31–40; E. A. Bender, Discrete Math. 8 (1974), 219–226; I. P. Goulden and D. M. Jackson, Combinatorial Enumeration (New York: Wiley, 1983), exercise 5.5.2; M.-P. Delest and G. Viennot, Theoretical Comp. Sci. 34 (1984), 169–206; W.-J. Woan, L. Shapiro, and D. G. Rogers, AMM 104 (1997), 926–931; P. Flajolet and R. Sedgewick, Analytic Combinatorics (2009), 660–662.]

304. E. D. Demaine and M. L. Demaine [Graphs and Combinatorics 23 (2007), Supplement, 195–208] show the NP-completeness also of several other related problems, such as to exactly pack given boxes of sizes {1 × x_1, ... , 1 × xn} into a given rectangle.

305. A scheme of “even/odd coordinates” (see exercise 145 and answer 133) works beautifully to represent the space occupied by a windmill domino: Encode the large square in row i and column j by the ordered pair (2i+1)(2j+1); encode the small “tilted” square that overlaps two adjacent large squares by the midpoint between them. Then, for example, ‘15’ is the large square in row 0 and column 2; ‘25’ is the small tilted square whose top and bottom halves are the bottom and top quarters of 15 and 35. Large squares have area 4; small tilted squares have area 2; the encoding of each square specifies the coordinates of its center point. The relevant coordinates xy in an m×n box satisfy 0 < x < 2n and 0 < y < 2m, where x and y are integers that aren’t both even.

Therefore the possible placements of the leftmost windmill domino are either {13, 15, 12, 23} + (2k, 2l), {33, 53, 23, 32} + (2k, 2l), {33, 31, 34, 23} + (2k, 2l), or {31, 11, 41, 32} + (2k, 2l), where k and l are nonnegative integers.

(a) Here it suffices to use a 5 × 5 box, and to require that the small squares of each option are either {34, 45}, {47, 56}, {76, 65}, or {63, 54}. Each piece has exactly four such options; for example, if we call the leftmost piece ‘0’, its options are ‘0 35 37 34 45’, ‘0 57 77 47 56’, ‘0 53 33 63 54’, ‘0 75 73 76 65’. The problem has 4 · 183 solutions, in groups of four that are related by 90° rotation. Here are six of the eight classes of equivalent solutions whose large squares form a symmetric shape:

(b) Algorithm X quickly finds 501484 = 2 · 4 + 4 · 125369 solutions, including four classes that are symmetric under reflection and 125369 unsymmetric classes. One of the symmetric examples is shown below; also one of the 164 asymmetric classes whose small squares do at least form a symmetric shape.

(c) The 288 = 2 · 4 + 4 · 70 solutions include four symmetric classes (like the one shown) and 70 that have no symmetry.

(d) We can set this up as a 7 × 7 problem in which the small squares form a rectangle whose corners are {47, 74, 8b, b8}. It has 2 · 2696 solutions, all asymmetric; 2 · 95 of them fit in a 5 × 5 box, and 2 · 3 of them have large squares that form the symmetric shape shown.

(e) Now there are two possibilities: We might have an 8 × 8 box, with small squares in the rectangle whose corners are {34, 43, cd, dc}; or we might have a 9 × 9 box, with small squares confined to the rectangle {45, 54, de, ed}. The first case has 69120 = 2 · 4 + 4 · 17278 solutions, four with reflective symmetry; the second case has a whopping 157398 = 2 · 75 + 4 · 39312 solutions, with 75 classes unchanged under reflection. Symmetric solutions of both types are shown.

306. Introduce items 0 to 9 and xy as in the previous answer, as well as pxy and #xy; again x and y aren’t both even, and 0 < x < 2n, 0 < y < 2m. Here pxy and #xy are primary, but the xy items are secondary. Options of the first kind, like ‘0 p35 35:1 p37 37:1 p34 34:1 p45 45:1’, specify placement of a piece. Options of the second kind, ‘pxy xy:0’, allow square xy to be empty. Options of the third kind, either ‘#xy xy:0’ or ‘#xy xy:1 (x−2)y:a (x+2)y:b x(y−2):c x(y+2):d’ for binary variables a, b, c, d with a + b + c + d = 2, and where both x and y are odd, enforce the snake condition for large squares. Options of the fourth kind, either ‘#xy xy:0’ or ‘#xy xy:1 (x−1)(y−1):a (x−1)(y+1):b (x+1)(y−1):c (x+1)(y+1):d’ and where x + y is odd, enforce the snake condition for small squares. Nonsharp branching (exercise 10) should be used.

Those options unfortunately produce a huge number of spurious solutions containing 4-cycles. One can rule out the 4-cycle whose large squares have a given x′ y′ as midpoint by using Algorithm M and introducing a new primary item #x′ y′ whose multiplicity is [0 .. 3]. (Notice that x′ and y′ are both even.) This primary item is appended to every option of type 3 that begins with ‘#xy xy:1’, where xy is one of the four squares touching point x′y′. The 4-cycles of small squares can be ruled out similarly, with new primary items #xy!, where x + y is even.

Every snake-in-the-box cycle of 20 large squares will fit into a box of size 3 × 9, 4×8, 5×7, or 6×6; and Algorithm M finds respectively (0, 0, 4·9, 8·8) solutions in those four cases. Six of the eight 6 × 6 equivalence classes are, however, spurious solutions, because their small squares form an 8-cycle and a 12-cycle instead of a single 20-cycle. Thus there are eleven essentially different solutions. Two of each size are shown below. [The middle two examples show two of the large squares touching at a corner. The definition of snake-in-the-box cycles allows this to happen; but five of the eleven solutions don’t have this “defect.” See Cubism For Fun 41 (October 1996), 30–32.]

307. “Factoring” with the residues (i − j) mod 3 and (i + j) mod 3, we see that the domino must go into adjacent cells with (i − j) mod 3 ≠ 1 and (i + j) mod 3 ≠ 2. That means either {(3i, 3j), (3i, 3j + 1)} or {(3i + 1, 3j + 2), (3i + 2, 3j + 2)}. Conversely, it’s easy to insert straight trominoes after placing a domino into any of those cell pairs.

308. (a) Each shape now has integer pairs of the forms (x, y) and (x, y)′. One elementary transformation, which rotates by 60°, takes (x, y) ↦ (x + y, − x)′ and (x, y)′ ↦ (x+y+1,−x); the shape’s triangles should be shifted afterwards so that all coordinates are nonnegative and as small as possible. The other elementary transformation, which is a reflection, simply takes (x, y) ↦ (y, x) and (x, y)′ ↦ (y, x)′.

For convenience, let’s write just xy for (x, y). One tetriamond is the triangle of size 2, {00, 01, 10, 00′}. It has two base placements; the other one is {01′, 10′, 11′, 11}. Another tetriamond is “straight,” {00, 00′, 10, 10′}, and it has six base placements. (Three of them, such as {00, 00′, 01, 01′}, involve reflection; hence that tetriamond has two one-sided versions.) The remaining tetriamond is “bent,” {00′, 01, 10, 10′}, a hexagon minus a diamond. Its six base placements are all obtained by rotation.

(b) Four of the 20-iamonds are convex, namely those parameterized by (6, 4, 0, 0), (10, 10, 1, 0), (4, 2, 1, 0), and (5, 5, 2, 0) in the notation of exercise 143. But only (4, 2, 1, 0) can be packed with the four pentiamonds — in fact in two ways, differing by a bipair.

(c) The convex 30-iamonds (15, 15, 1, 0) and (7, 7, 1, 1) cannot be packed. But (4, 2, 1, 1), (5, 5, 3, 0), (3, 3, 3, 1) have respectively 3, 1, and 4 distinct solutions.

309. (a) (A, ... , L) have respectively (6, 3, 6, 1, 6, 6, 12, 12, 6, 12, 12, 12) placements.

(The hexiamonds have also been given descriptive names: A = lobster (or heart); B = butterfly (or spool); C = chevron (or bat); D = hexagon; E = crown (or boat); F = snake (or wave); G = hook (or shoe); H = signpost (or pistol or airplane); I = bar (or rhomboid); J = crook (or club or ladle); K = yacht (or steps); L = sphinx (or funnel).)

(b) Hexiamonds K and L are special, because they contain four triangles of one kind (∆ or ∇) and two of the other (∇ or ∆). The other hexiamonds are balanced, with three of each kind.

Eleven convex polygons are 72-iamonds, by exercise 143. Those with height less than 4, namely (36, 36, 1, 0), (19, 17, 0, 0), (18, 18, 2, 0), and (12, 12, 3, 0), are unsolvable. So is (9, 3, 0, 0), which is out of balance by 6. The other six are solvable; for example,

The shape (6, 2, 2, 1) is out of balance by 4. Consequently we can restrict K and L to about half of the positions where they would otherwise fit. The running time to find all solutions (without removing symmetry) thereby decreases, from 168 Gμ to 135 Gμ; thus the parity theory helps here, but not as much as might be expected.

What about the one-sided hexiamonds (with “flipped” versions of F through L, making 19 in all)? There are six convex polygons made up of 6 · 19 = 114 triangles, and again the small-height ones (57, 57, 1, 0), (28, 28, 1, 1), (19, 19, 3, 0) are unsolvable. The case (13, 9, 1, 0) has 1,687,429 solutions (found by Algorithm X in 11 Tμ). Shape (8, 8, 3, 3) has 4,790,046 distinct solutions (103 Tμ); (9, 5, 2, 1) has 17,244,919 (98 Tμ).

Historical notes: T. Scrutchin [U.S. Patent 895114 (1908)] described an early puzzle based on assembling checkered polyiamonds of sizes 3–7 into a large equilateral triangle. The complete set of hexiamonds was perhaps first invented by Charles H. Lewis, who submitted a paper about them to the American Mathematical Monthly in April 1958. His paper wasn’t judged worthy of publication; but a copy survives in the files of Martin Gardner, to whom he had sent a preprint. (He’d been inspired by Martin’s exposition of polyominoes in December 1957.) Lewis named his pieces hexotinoes, and said that they belonged to the family of “polotinoes,” which began with the monotino, the dotino, the trotino, three tetrotinoes, and four pentotinoes. He knew the parity rule, and he exhibited one of the ways to pack all 12 hexotinoes into a 6 × 6 rhombus.

Other people came up with similar ideas independently a few years later. It was T. H. O’Beirne who coined the names “polyiamond” and “hexiamond” — to the eternal dismay of language purists — first in letters to Richard Guy in 1960, then in his popular weekly columns in New Scientist [12 (1961), 261, 316–317, 379, 706–707]. He introduced an intriguing problem about packing the one-sided hexiamonds into the rosette shape formed by 19 hexagons (12 surrounding 6 surrounding 1); see pages 452–455 of FGbook for details. Martin Gardner wrote about the subject in Scientific American 211, 6 (December 1964), 123–130, and hexiamonds were soon sold as pleasing puzzles in Japan, Germany, the USA, and elsewhere. The 24 heptiamonds also have many aficionados, but they are beyond the scope of this book.

The earliest papers about hexiamonds considered mostly standard shapes like parallelograms, or shapes that are decidedly non-convex. Polygon (6, 2, 2, 1) above, the “diaper,” may have first appeared as problem 130 in the Russian magazine Nauka i Zhizn’ #6 (1969), 146; #7 (1969), 101; Michael Beeler enumerated its solutions in HAKMEM (M.I.T. A.I. Laboratory, 1972), Hack 112. Polygon (6, 6, 3, 2) has apparently not occurred previously in print, although it has more solutions than the others.

310. The container holds 4m+2 triangles; m = 18 doesn’t work, so we need at least six empty cells. The author’s favorite way constrains them to be well-separated “teeth”:

311. H. Postl found a nice proof that N must be at least 190: Replace hexiamonds A, G, K by the heptiamond that includes a hexagon. The twelve resulting pieces contain 75 triangles; enlarge them by appending quarter-size triangles around all the edges. This adds 91 trapezoids and 163 quarter-triangles. The latter must occupy at least 91 + (163 − 91)/3 = 115 triangles, because we can’t fill a triangle without using a trapezoid.

Exercise 7–137 explains how to obtain many generalized toruses that are composed of 95 rhombuses; so we might as well make the repeating pattern as square as possible by choosing (a, b, c, d) = (11, −4, −1, 9), as in the solution below. There are (astonishingly) 321530 such packings, each of which represents 24 different solutions when the heptiamonds revert to {A, G, K}. The example shown is one of only 1768 solutions for which the three resulting “females” attract three neighboring “males.”

[The smallest region for pentomino wallpaper has 143 cells. See A. Thoen and A. van de Wetering, Facets of Pentominoes (2018), 95.]

312. Adrian Struyk wrapped the octahedron with hexiamonds, and showed it to Martin Gardner in 1964. An attractive solution by Walter Stead (1970, unpublished),

doesn’t bend any piece in more than two places. (Incidentally, Thijs Notenboom showed in 1967 how to wrap the icosahedron with the four pentiamonds.)

313. The whirled versions of pieces (A, ... , L) can be packed in respectively (13, 2 · 2, 10, 6 · 55, 19, 2 · 10, 9, 10, 2 · 10, 18, 6, 20) ways. But with flipped whirls, the one-sided pieces lead to different shapes, and the counts for (F, G, ... , L) change to (2 · 6, 7, 8, 2 · 0, 25, 7, 8). Here’s how the pattern of answer 310 looks when scaled up by 12:

[The “whirl” in this exercise is the case n = 3 of an n-whirl, which has n^2 + 3 triangles for n ≥ 2. In 1936, Maurits Escher visited the Alhambra and saw a pattern related to the whirl tessellation. He was subsequently inspired to develop it much further; see The World of M. C. Escher (1971), plates 84 and 199.]

314. To make the same shape from two pairs {a, b} and {c, d} of polyiamonds (or polyominoes, etc.), choose an n-celled region A into which any solution will fit. Use four primary items {a, b, c, d} and 6n secondary items 0α, 1α, aα, bα, cα, dα for each cell α. For each placement ‘a α_1 ... αs’ in A, and each of the 2s sequences q_1 ... qs with qk ∈ {c, d}, create the option ‘a 0α_1 q_1α_1 ... 0αs qsαs aβ_1 ... aβn−s’, where {β_1,...,βn−s} = A \ {α_1,...,αs}. Also create similar options for each placement of b, c, d, with the roles of (0, a, c, d) replaced respectively by (0, b, c, d), (1, c, a, b), (1, d, a, b).

Choose one of {a, b, c, d} (one-sided if possible) and restrict it to a single placement. For the pentiamond problem, the author chose the piece a that includes a tetrahedron, and placed it in the center of a 70-iamond A. There are three separate cases, depending on which piece is called b; they yielded three huge exact cover problems, each of which had 15300 options of length 76 (thus total length 1.2 million). Yet Algorithm X solved each problem in at most 1.5 Gμ, including 0.3 Gμ just to load the data.

The answer, as Sicherman observed, is unique. [See Ed Pegg Jr.’s blog, www.mathpuzzle.com/30November2008.xhtml. Solomon Golomb, in Recreational Math. Mag. #5 (October 1961), 3–12, had shown that the twelve pentominoes can be partitioned into three sets of four, each of which make congruent pairs.]

315. Proceed as in answer 308, but simply let (x, y) ↦ (x + y, xmax − x); ignore (x, y)′.

[There’s also an even/odd coordinate system for hexagons, with hexagon xy represented by (2x + 1, 2y + 1), and the edge between adjacent hexagons represented by their average. Then 60° rotation takes (x, y) ↦ (x + y − 1, xmax − x + 1).]

316. There are 12 · 12290 solutions, and it’s not hard to find one by hand. (The first solutions were discovered independently by T. Marlow and E. Schwartz in 1966; the total number was found by K. Noshita in 1974.) The example shown here has the trihexes “maximally separated.” [The seven tetrahexes pack the rhomboid {xy | 0 ≤ x < 4, 0 ≤ y < 7} in 2 · 9 ways, and the skew triangle {xy | 0 ≤ x < 7, x ≤ y < 7} in 2 · 5 ways; but they can’t pack the triangle {xy | 0 ≤ x < 7, 0 ≤ y < 7 − x}.]

317. The scaled-up “bar,” “wave,” and “propeller” cannot be packed. But the “bee,” “arch,” “boot,” and “worm” are doable in respectively 2 · 2, 1, 10, and 4 ways, such as

[This problem was introduced by E. Schwartz in 1966 and independently by G. Edgar in 1967, who showed their solutions to Martin Gardner. Edgar pointed out that the rosettes can actually be placed in two ways — either rising or falling slightly from left to right when put together. The three one-sided tetrahexes therefore lead to distinct scaled-up shapes. Only one of those two is packable, for the boot and the worm; both are impossible for the wave. The slight tilting accounts for some of the remarkable properties of R. W. Gosper’s “flowsnake” fractal; see M. Gardner, Scientific Amer. 235, 6 (December 1976), 124–128, 133; A. Vince, SIAM J. Discrete Math. 6 (1993), 501–521.]

318. The “holes” in the T-grid correspond to vertices of the infinite triangular grid; and every hexagon of the T-grid is inside exactly one of the triangles made by those vertices. More formally, we can let

Δ (x,y)↔hexagon (x−y,x+2y+1);∇ (x,y)′↔hexagon (x−y,x+2y+2).

Adjacent triangles correspond to adjacent hexagons. The hexiamond hexahexes are

319. One way is to replace each square by a 3 × 3 array, representing by . But it uses only 4 pixels out of 9. A more compact scheme is able to use 4 pixels out of every 8: We rotate the pieces by 45° and represent by , separated by . For example, the 14 tetraboloes take the following forms:

This scheme sets up a one-to-one correspondence between n-aboloes and 2n-ominoes on the “H-grid,” which is the set of all pixels (x, y) with ⌊x/2⌋ + ⌊y/2⌋ even. (Each 2n-omino is kingwise connected; it actually consists of n dominoes.)

Formally speaking, let’s divide every square cell into four quarters, by cutting at the diagonals. Then every n-abolo occupies 2n quarters; and the (north, east, south, west) quarters of cell (x, y), in polyabolo coordinates, correspond respectively to cells (2x − 2y, 2x + 2y) + ((0, 1), (1, 1), (1, 0), (0, 0)) of the H-grid.

[After first seeing the H-grid versions of the tetraboloes, the author felt a foolish but irresistible urge to pack them into a 10 × 12 box, putting seven of them in the H-grid and the other seven in the complementary H-grid, leaving eight vacant pixels at the sides. This corresponds to putting the tetraboloes into two layers of a certain frame that’s capable of holding 29 halfsquares. It turned out that there are 8 · 305 ways to do this (found by Algorithm X in 10 Gμ). For example:

Nowadays, polyaboloes are often called “polytans,” based on their connection to classical tangram puzzles from 18th-century China. T. H. O’Beirne introduced polyaboloes in New Scientist 13 (18 January 1962), 158–159.]

320. Every convex polyabolo can be characterized by six more-or-less independent parameters: We start with an m × n rectangle, then cut off triangles of sizes a, b, c, d at the lower left, lower right, upper right, and upper left corners, where a+b ≤ n, b+c ≤ m, c + d ≤ n, and d + a ≤ m. The number of halfsquares is N = 2mn − a^2 − b^2 − c^2 − d^2. To avoid duplicates, we require m ≤ n, and insist that (a, b, c, d) be lexicographically greater than or equal to (b, a, d, c), (c, d, a, b), (d, c, b, a). Furthermore, if m = n, this 4-tuple (a, b, c, d) should also be lexicographically greater than or equal to (a, d, c, b), (b, c, d, a), (c, b, a, d), (d, a, b, c).

The smallest positive area achievable with m < n is 2m(n − m) halfsquares; and when m = n the smallest is 2n − 1. Thus we must have n ≤ (N + 2)/2, and it’s feasible to backtrack through a finite number of cases.

There are 63 solutions when N = 56. But most of them are unpackable, because of an important property noted by T. H. O’Beirne in 1962: Exactly five of the tetraboloes, namely {E, G, J, K, L}, have an odd number of unmatched 2 sides in each direction. It follows that a + c (and b + d) must be odd.

Just 10 of the 63 solutions pass this extra test. Two of those ten — (1×29; 1, 1, 0, 0) and (3×11; 3, 1, 0, 0) — don’t work. But the other eight are achievable:

Most of them were cracked by E. S. Ainley in 1965; but H. Picciotto found ‘*’ in 1989.

[This enumeration problem was first studied by F. T. Wang and C.-C. Hsiung, AMM 49 (1942), 596–599, who proved that there are 20 convex 16-aboloes. The totals for general N are OEIS sequence A245676, contributed by E. Fox-Epstein in 2014.]

321. [In a letter to Martin Gardner dated 12 March 1967, O’Beirne said that he now knew of 13 solutions, with help from several readers. “Are these the lot?” The answer is yes: The total is indeed 13. The solution shown here leads to three of the others, via tricky rearrangements.]

322. (i) We can reduce polysticks to (disconnected) polyominoes, by 3-fold enlargement: Let vertex ij of a square grid correspond to pixel (3i)(3j); and let the line segment between adjacent vertices ij — i′j′ correspond to the two pixels between (3i)(3j) and (3i′)(3j′). Placements can intersect each other only at internal pixels where two parallel segments touch; we can prevent crossing by making such pixels secondary.

For example, to pack the 6 × 6 array in the example, we use the pixels xy for 0 ≤ x, y ≤ 18, where x or y is a multiple of 3; item xy is secondary if 3 divides both x and y. One of the options for the T-shaped tetrastick is ‘04 05 07 08 16 26 36 46 56’; one of the options for the V-shaped tetrastick is ‘34 35 36 37 38 49 59 69 79 89’. The secondary item 36 ensures that these options won’t both be chosen simultaneously.

(ii) Instead of scaling up by 3, we can scale up by 2, as in the even/odd coordinate system, by letting vertex ij correspond to pixel (2i)(2j). Then segment ij — i′j′ corresponds to pixel (i + i′)(j + j′); and the 6 × 6 example involves primary items xy for 0 ≤ x, y ≤ 12 with x + y odd, together with secondary items xy with x and y both even. The example T and V options in this scheme become ‘03 05 14 24 34’ and ‘23 24 25 36 46 56’; now it’s the secondary item 24 that keeps them from interacting.

Scheme (i) can be used without change to answer 266. Scheme (ii) is almost twice as fast; but answer 266 must then be modified so that it never shifts by odd amounts. (Notice, for example, that the O and X tetrasticks each have only one base placement in scheme (ii), namely ‘01 10 12 21’ and ‘12 21 23 32’. Shifting by 11 would change O to X and vice versa!) Thus, 90° rotation must be redefined as (x, y) ↦(y, xmax + (xmax & 1) − x), in the modified answer 266; also, δx and δy must be even.

[Polysticks were named and explored by B. R. Barwell in JRM 22 (1990), 165– 175. They had actually been studied in the 1940s by H. D. Benjamin and T. R. Dawson, who already knew how to pack the pieces for n ≤ 4 into a 6 × 6 grid; see G. P. Jelliss, JRM 29 (1998), 140–142. See also FGbook, pages 457–472.]

323. (a) For example, the vertices (m, n) of an ordinary square grid can be skewed to

(m,n)′=(m−(n mod 2)ϵ,n−(m mod 2)ϵ),where ϵ is the degree of skew.

Notice that each square of the skewed grid has a clockwise or counterclockwise “spin.”

(b) There’s a nice way to represent each square as a 5-pixel cross, and each rhombus as a 3-pixel diagonal. For example, here are pixel equivalents of the tetraskews:

(Lowercase letters indicate the rhombuses here only for clarity; all pixels are either “in” or “out.” The shapes fit together only when squares and rhombuses alternate properly.)

(c) The 4 × 10 frame in the example has 486 solutions; the analogous 5 × 8 frame has 572; these were first enumerated by Brendan Owen in 2000. There are 3648 ways to fit the pieces into a 2 × 21 frame, but 2 × 20 is too tight.

However, those counts can be divided by 2, because solutions to this problem come in pairs. Consider an arrangement of ten unskewed tetrominoes that involves one square, one straight, two skews, two tees, and four ells. It can be skewed in four ways, because we have two choices for which cells should be rhombuses and two choices for the spins; and it will be a valid skewed solution if and only if the resulting ten tetraskews are distinct. Changing the spins of a valid solution always gives another valid solution in which K ↔ L, S ↔ Z, U ↔ V are swapped. Every solution therefore has a dual, which looks rather different but is well defined.

For example, the 486 solutions to the 4 × 10 rectangle problem correspond to exactly 226 unskewed arrangements that are distinct under reflections, 17 of which actually yield two dual pairs of skewed solutions, in which the roles of squares and rhombuses are reversed! Here’s one such case:

[Michael Keller named the polyskews in 1993, and found a way to pack the tetraskews into two 4×5 frames, thus solving the 4×10 and 5×8 rectangles simultaneously. (See World Game Review 12 (1994), 12. That problem has just 24 solutions.) Generalizations to 3D await investigation.]

References: Polyforms live on many excellent and well-illustrated websites — notably puzzler.sourceforge.net by David Goodger; www.polyforms.eu by Peter Esser; www.iread.it/lz/polymultiforms2.xhtml by Livio Zucca; userpages.monmouth.com/~colonel/polycur.xhtml by George Sicherman; www.recmath.org/PolyPages/ by Andrew Clarke; abarothsworld.com/Puzzles.htm by Abaroth. In particular, Abaroth’s page “Squaring the Hexagon” discusses many ways to reduce one polyform to another. See also Ed Pegg Jr.’s chapter in Tribute to a Mathemagician (2005), 119–125.

324. The same ideas apply, but with three coordinates instead of two, and with the elementary transformations (x, y, z) ↦ (y, xmax − x, z), (x, y, z) ↦ (y, z, x).

Pieces (1, 2, ... , 7) have respectively (12, 24, 12, 12, 12, 12, 8) base placements, leading to 144 + 144 + 72 + 72 + 96 + 96 + 64 options for the 3 × 3 × 3 problem.

325. It’s tempting, but wrong, to try to compute the Somap by considering only the 240 solutions that restrict the tee and the claw as suggested in the text; the pairwise semidistances between these special solutions will miss many of the actual adjacencies. To decide if u — v, one must compare u to the 48 solutions equivalent to v.

(a) The strong Somap has vertex degrees 7^1 6^7 5^19 4^31 3^59 2^63 1^45 0^15; so an “average” solution has (1 · 7 + 7 · 6 + … + 15 · 0)/240 ≈ 2.57 strong neighbors. (The unique vertex of degree 7 has the level-by-level structure from bottom to top.) This graph has two edges between and , so it’s actually a multigraph.

The full Somap has vertex degrees 21^2 18^1 16^9 15^13 14^10 13^16 12^17 11^12 10^16 9^28 8^26 7^25 6^26 5^16 4^17 3^3 2^1 1^1 0^1, giving an average degree ≈ 9.14. (Its unique isolated vertex is , and its only pendant vertex is . Two other noteworthy solutions, and , are the only ones that contain the two-piece substructure . There are 14 instances of repeated edges.)

(b) The Somap has just two components, namely the isolated vertex and the 239 others. The latter has just three bicomponents, namely the pendant vertex, its neighbor, and the 237 others. Its diameter is 8 (or 21, if we use the edge lengths 2 and 3).

The strong Somap has a much sparser and more intricate structure. Besides the 15 isolated vertices, there are 25 components of sizes {8 × 2, 6 × 3, 4, 3 × 5, 2 × 6, 7, 8, 11, 16, 118}. Using the algorithm of Section 7.4.1.2, the large component breaks down into nine bicomponents (one of size 2, seven of size 1, the other of size 109); the 16-vertex component breaks into seven; and so on, totalling 58 bicomponents altogether.

(One can also consider “physical” Somaps with 480 vertices, by saying that solutions are equivalent under rotation but not reflection. There are no repeated edges. The degree sequences are 7^2 6^14 ... 0^30 and 21^4 18^2 ... 0^2, double what we had before.)

[The Somap was first constructed by R. K. Guy, J. H. Conway, and M. J. T. Guy, without computer help. It appears on pages 910–913 of Berlekamp, Conway, and Guy’s Winning Ways, where all of the strong links are shown, and where enough other links are given to establish near-connectedness. Each vertex in that illustration has been given a code name; for example, the seven special solutions mentioned in part (a) have code names B5f, W4e, W2f, R7d, LR7g, YR3a, and R3c, respectively.]

326. Let the cubie coordinates be 51z, 41z, 31z, 32z, 33z, 23z, 13z, 14z, 15z, for z ∈ {1, 2, 3}. Replace matrix A of the exact cover problem by a simplified matrix A′ having only items (1, 2, 3, 4, 5, 6, 7, S), where S is the sum of all items xyz of A where x · y · z is odd. Any solution to A yields a solution to A′ with item sums (1, 1, 1, 1, 1, 1, 1, 10). But that’s impossible, because the S counts of pieces (1, ... , 7) are at most (1, 2, 2, 1, 1, 1, 1). [See the Martin Gardner reference in answer 333.]

327. (a) The solution counts, ignoring symmetry reduction, are: 4 × 5 corral (2), gorilla (2), smile (2), 3 × 6 corral (4), face (4), lobster (4), castle (6), bench (16), bed (24), doorway (28), piggybank (80), five-seat bench (104), piano (128), shift 2 (132), 4 × 4 coop (266), shift 1 (284), bathtub (316), shift 0 (408), grand piano (526), tower 4 (552), tower 3 (924), canal (1176), tower 2 (1266), couch (1438), tower 1 (1520), stepping stones (2718). So the 4 × 5 corral, gorilla, and smile are tied for hardest, while stepping stones are the easiest. (The bathtub, canal, bed, and doorway each have four symmetries; the couch, stepping stones, tower 4, shift 0, bench, 4 × 4 coop, castle, five-seat bench, piggybank, lobster, piano, gorilla, face, and smile each have two. To get the number of essentially distinct solutions, divide by the number of symmetries.)

(b) Notice that the stepping stones, canal, bed, and doorway appear also in (a). The solution counts are: W-wall (0), almost W-wall (12), bed (24), apartments 2 (28), doorway (28), clip (40), tunnel (52), zigzag wall 2 (52), zigzag wall 1 (92), underpass (132), chair (260), stile (328), fish (332), apartments 1 (488), goldfish (608), canal (1176), steps (2346), stepping stones (2718); hence “almost W-wall” is the hardest of the possible shapes. Notice that the stepping stones, chair, steps, and zigzag wall 2 each have two symmetries, while the others in Fig. 75(b) all have four. The 3×3×3 cube, with its 48 symmetries, probably is the easiest possible shape to make from the Soma pieces.

[Piet Hein himself published the tower 1, shift 2, stile, and zigzag wall 1 in his original patent; he also included the bathtub, bed, canal, castle, chair, steps, stile, stepping stones, shift 1, five-seat bench, tunnel, W-wall, and both apartments in his booklet for Parker Brothers. Parker Brothers distributed four issues of The SOMA® Addict in 1970 and 1971, giving credit for new constructions to Noble Carlson (fish, lobster), Mrs. C. L. Hall (piano, clip, underpass), Gerald Hill (towers 2–4), Craig Kenworthy (goldfish), John W. M. Morgan (piano, face, gorilla, smile), Rick Murray (grand piano), and Dan Smiley (doorway, zigzag wall 2). Sivy Farhi published a booklet called Somacubes in 1977, containing the solutions to more than one hundred Soma cube problems including the bench, the couch, and the piggybank.]

328. By eliminating symmetries, there are (a) 421 distinct cases with cubies omitted on both layers, and (b) 129 with cubies omitted on only one layer. All are possible, except in the one case where the omitted cubies disconnect a corner cell. The easiest of type (a) omits {000, 001, 200} and has 3599 solutions; the hardest omits {100, 111, 120} and has 2·45 solutions. The easiest of type (b) omits {000, 040, 200} and has 3050 solutions; the hardest omits {100, 110, 140} and has 2 · 45 solutions. (The two examples illustrated have 2 · 821 and 4 · 68 solutions. Early Soma solvers seem to have overlooked them!)

329. (a) The 60 distinct cases are all quite easy. The easiest has 3497 solutions and uses {002, 012, 102} on the top level; the hardest has 268 solutions and uses {002, 112, 202}.

(b) Sixteen of the 60 possibilities are disconnected. Three of the others are also impossible — namely those that omit {01z, 13z, 21z} or {10z, 11z, 12z} or {10z, 11z, 13z}. The easiest has 3554 solutions and omits {00z, 01z, 23z}; the hardest of the possibles has only 8 solutions and omits {00z, 12z, 13z}.

(The two examples illustrated have 2 · 132 and 2 · 270 solutions.)

330. T. Bundgård and C. McFarren found in 1999 that all but 216 are realizable [www.fam-bundgaard.dk/SOMA/NEWS/N990308.HTM]. Five cases have unique (2 · 1) solutions:

331. Every polycube has a minimum enclosing box for which it touches all six faces. If those box dimensions a×b×c aren’t too large, we can generate such polycubes uniformly at random in a simple way: First choose 27 of the abc possible cubies; try again if that choice doesn’t touch all faces; otherwise try again if that choice isn’t connected.

For example, when a = b = c = 4, about 99.98% of all choices will touch all faces, and about 0.1% of those will be connected. This means that about .001(6427)≈8×1014 of the 27-cubie polycubes have a 4 × 4 × 4 bounding box. Of these, about 5.8% can be built with the seven Soma pieces.

But most of the relevant polycubes have a larger bounding box; and in such cases the chance of solvability goes down. For example, ≈ 6.2 × 10^18 cases have bounding box 4 × 5 × 5; ≈ 3.3 × 10^18 cases have bounding box 3 × 5 × 7; ≈ 1.5 × 10^17 cases have bounding box 2 × 7 × 7; and only 1% or so of those cases are solvable.

Section 7.2.3 will discuss the enumeration of polycubes by their size.

332. Each interior position of the penthouse and pyramid that might or might not be occupied can be treated as a secondary item in the corresponding exact cover problem. We obtain 2 · 10 solutions for the staircase; (223, 8 · 286) solutions for the penthouse with hole at the (bottom, middle); and 2 · 32 solutions for the pyramid, of which 2 · 2 have all three holes on the diagonal and 2 · 3 have no adjacent holes.

333. A full simulation of gravity would be quite complex, because pieces can be prevented from tipping with the help of their neighbors above and/or at their side. If we assume a reasonable coefficient of friction and an auxiliary weight at the top, it suffices to define stability by saying that a piece is stable if and only if at least one of its cubies is immediately above either the floor or a stable piece.

The given shapes can be packed in respectively 2 · 202, 2 · 21, 2 · 270, 8 · 223, and 2 · 122 ways, of which 2 · 202, 2 · 8, 2 · 53, 8 · 1, and 2 · 6 are stable. Going from the bottom level to the top, the layers give a decently stable cot; a fragile vulture comes from ; a delicate mushroom comes from ; and a delicate cantilever from . The author’s cherished set of Skjøde Skjern Soma pieces, made of rosewood and purchased in 1967, includes a small square base that nicely stabilizes both mushroom and cantilever. The vulture needs a book on top.

[The casserole and cot are due respectively to W. A. Kustes and J. W. M. Morgan. The mushroom, which is hollow, is the same as B. L. Schwartz’s “penthouse,” but turned upside down; John Conway noticed that it then has a unique stable solution. See Martin Gardner, Knotted Doughnuts (1986), Chapter 3.]

334. Infinitely many cubies lie behind a wall; but it suffices to consider only the hidden ones whose distance is at most 27 − v from the v visible ones. For example, the W-wall has v = 25, and the two invisible cubies are {332, 331} if we use the coordinates of answer 326. We’re allowed to use any of {241, 242, 251, 252, 331, 332, 421, 422, 521, 522} at distance 1, and {341, 342, 351, 352, 431, 432, 531, 532, 621, 622} at distance 2. (The stated projection doesn’t have left-right symmetry.) The X-wall is similar, but it has v = 19 and potentially (9, 7, 6, 3, 3, 2, 1) hidden cubies at distances 1 to 7 (omitting cases like 450, which is invisible at distance 2 but “below ground”).

Using secondary items for the optional cubies, we must examine each solution to the exact cover problem and reject those that are disconnected or violate the gravity constraint of exercise 333. Those ground rules yield 282 solutions for the W-wall, 612 for the X-wall, and a whopping 1,130,634 for the cube itself. (These solutions fill respectively 33, 275, and 13842 different sets of cubies.) Here are examples of some of the more exotic shapes that are possible, as seen from behind and below:

There also are ten surprising ways to make the cube façade if we allow hidden “underground” cubies: The remarkable construction raises the entire cube one level above the floor, and is gravitationally stable, by exercise 333’s criteria! Unfortunately, though, it falls apart — even with a heavy book on top.

[The false-front idea was pioneered by Jean Paul Francillon, whose construction of a fake W-wall was announced in The SOMA® Addict 2, 1 (spring 1971).]

335. (a) Each of 13 solutions occurs in 48 equivalent arrangements. To remove the symmetry, place piece 7 horizontally, either (i) at the bottom or (ii) in the middle. In case (ii), add a secondary ‘s’ item as in answer 268, and append ‘s’ also to all placements of piece 6 that touch the bottom more than the top. Run time: 400 Kμ.

[This puzzle was number 3–39 in Hoffmann’s Puzzles Old and New (1893). Another 3 × 3 × 3 polycube dissection of historical importance, “Mikusinski’s Cube,” was described by Hugo Steinhaus in the 2nd edition of his Mathematical Snapshots (1950). That one consists of the ell and the two twist pieces of the Soma cube, plus the pentacubes B, C, and f of exercise 340; it has 24 symmetries and just two solutions.]

(b) Yes: Michael Reid, circa 1995, found the remarkable set

which also makes 9 × 3 × 1 uniquely(!). George Sicherman carried out an exhaustive analysis of all relevant flat polyominoes in 2016, finding exactly 320 sets that are unique for 3 × 3 × 3, of which 19 are unique also for 9 × 3 × 1. In fact, one of those 19,

is the long-sought “Holy Grail” of 3 × 3 × 3 cube decompositions: Its pieces not only have flatness and double uniqueness, they are nested (!!). There’s also Yoshiya Shindo’s

known as the “Neo Diabolical Cube” (1995); notice that it has 24 symmetries, not 48.

336. This piece can be modeled by a polycube with 20 + 20 + 27 + 3 cubies, where we want to pack nine of them into a 9 × 9 × 9 box. Divide that box into 540 primary cells (which must be filled) and 189 secondary cells (which will contain the 27 cubies of the simulated dowels). Answer 324 now yields an exact cover problem with 1536 options; and Algorithm X needs only 33 Mμ to discover 24 solutions, all equivalent by symmetry. (Or we could modify answer 324 so that all offsets have multiples of 3 in each coordinate; then there would be only 192 options, and the running time would go down to 8 Mμ.) One packing is , with dowels at .

One might be tempted to factor this problem, by first looking at all ways to pack nine solid bent trominoes into a 3 × 3 × 3 box. That problem has 5328 solutions, found in about 5 Mμ; and after removing the 48 symmetries we’re left with just 111 solutions, into which we can try to model the holes and dowels. But such a procedure is rather complicated, and it doesn’t really save much time, if any.

Ronald Kint-Bruynseels, who designed this remarkable puzzle, also found that it’s possible to drill holes in the solid cubies, parallel to the other two, without destroying the uniqueness of the solution(!). [Cubism For Fun 75 (2008), 16–19; 77 (2008), 13–18.]

337. Let’s use even/odd coordinates as in exercise 145, so that each final face has one coordinate in {0, 6} and two coordinates in {1, 3, 5}. The first goal has red spots on faces 330, 105, 501, 015, 033, 051, 611, 615, 651, 655, 161, 165, 363, 561, 565, 116, 136, 156, 516, 536, 556. The other goal has green spots on 19 of those 21 faces; but 303 replaces 033 and 633 replaces 363. (For simplicity, we’ll ignore alternative setups; there are 16 ways to put spots on dice, not just two.)

Nine bent tricubes will pack a 3 × 3 × 3 cube in 5328 ways. (They fall into 111 equivalence classes of size 48, under rotation and reflection; but that fact is irrelevant here.) Take any such solution and color its 54 external faces with the red solution. Then see if its pieces can be rearranged to give the green solution.

Notice that each bent tricube has fourteen square faces; but the two “inner” faces are never visible in the final assembly. That assembly will specify from 2 to 7 of the 12 potential faces, leaving 5 to 10 faces unconstrained. Altogether we’ll have 21 faces specified red, 33 specified blank, and 54 still free.

It turns out that 371 of the 5328 red solutions can be rearranged into green solutions; in fact one case leads to 6048 different green solutions! And there are 52 combinations of red+green solutions that leave 18 faces unspecified, such as this:

We’re free to put anything we like on those 18 faces — giving red or green spots that are false clues, and/or concealing a third pattern that the puzzler is challenged to achieve.

(The classic “Spots Puzzle” in Hoffmann’s Puzzles Old and New (1893), No. 3–17, distributed by E. Wolff & Son’s pencil company, assembled a single die from straight tricubes. Lavery’s elegant “Twice Dice” was produced by Pentangle Puzzles in 1990.)

338. The straight tetracube and the square tetracube , together with the size-4 Soma pieces in (39), make a complete set.

We can fix the tee’s position in the twin towers, saving a factor of 32; and each of the resulting 40 solutions has just one twist with the tee. Hence there are five inequivalent solutions, and 256 · 5 altogether.

The double claw has 6 · 63 solutions. But the cannon, with 4 · 1 solutions, can be formed in essentially only one way. (Hint: Both twists are in the barrel.)

There are no solutions to ‘up 3’. But ‘up 4’ and ‘up 5’ each have 8 · 218 solutions (related by turning them upside down). Gravitationally, four of those 218 are stable for ‘up 5’; the stable solution for ‘up 4’ is unique, and unrelated to those four.

References: Jean Meeus, JRM 6 (1973), 257–265; Nob Yoshigahara, Puzzle World No. 1 (San Jose: Ishi Press International, 1992), 36–38.

339. All but 48 are realizable. The unique “hardest” realizable case, , has 2 · 2 solutions. The “easiest” case is the 2 × 4 × 4 cuboid, with 11120 = 16 · 695 solutions.

340. (a) A, B, C, D, E, F, a, b, c, d, e, f, j, k, l, ... , z. (It’s a little hard to see why reflection doesn’t change piece ‘l’. In fact, S. S. Besley once patented the pentacubes under the impression that there were 30 different kinds! See U.S. Patent 3065970 (1962), where Figs. 22 and 23 illustrate the same piece in slight disguise.)

Historical notes: R. J. French, in Fairy Chess Review 4 (1940), problem 3930, was first to show that there are 23 different pentacube shapes, if mirror images are considered to be identical. The full count of 29 was established somewhat later by F. Hansson and others [Fairy Chess Review 6 (1948), 141–142]; Hansson also counted the 35 + 77 = 112 mirror-inequivalent hexacubes. Complete counts of hexacubes (166) and heptacubes (1023) were first established soon afterwards by J. Niemann, A. W. Baillie, and R. J. French [Fairy Chess Review 7 (1948), 8, 16, 48].

(b) The cuboids 1×3×20, 1×4×15, 1×5×12, and 1×6×10 have of course already been considered. The 2 × 3 × 10 and 2 × 5 × 6 cuboids can be handled by restricting X to the bottom upper left, and sometimes also restricting Z, as in answers 268 and 270; we obtain 12 solutions (in 350 Mμ) and 264 solutions (in 2.5 Gμ), respectively.

The 3 × 4 × 5 cuboid is more difficult. Without symmetry-breaking, we obtain 3940 × 8 solutions in about 200 Gμ. To do better, notice that O can appear in four essentially different positions. With four separate runs we can find 5430/2 + 1348/4 + 716/2 + 2120/4 = 3940 solutions, in 35.7 + 10.0 + 4.5 + 7.1 ≈ 57 Gμ.

[The fact that solid pentominoes will fill these cuboids was first demonstrated by D. Nixon and F. Hansson, Fairy Chess Review 6 (1948), problem 7560 and page 142. Exact enumeration was first performed by C. J. Bouwkamp in 1967; see J. Combinatorial Theory 7 (1969), 278–280, and Indagationes Math. 81 (1978), 177–186.]

(c) Almost any subset of 25 pentacubes can probably do the job. But a particularly nice one is obtained if we simply omit o, q, s, and y, namely those that don’t fit in a 3 × 3 × 3 box. R. K. Guy proposed this subset in Nabla 7 (1960), 150, although he wasn’t able to pack a 5 × 5 × 5 at that time.

The same idea occurred independently to J. E. Dorie, who trademarked the name “Dorian cube” [U.S. Trademark 1,041,392 (1976)].

An amusing way to form such a cube is to make 5-level prisms in the shapes of the P, Q, R, U, and X pentominoes, using pieces {a, e, j, m, w}, {f, k, l, p, r}, {A, d, D, E, n}, {c, C, F, u, v}, {b, B, t, x, z}; then use the packing in answer 269(!). This solution can be found with six very short runs of Algorithm X, taking only 300 megamems overall.

Another nice way, due to Torsten Sillke, is more symmetrical: There are 70,486 ways to partition the pieces into five sets of five that allow us to build an X-prism in the center (with piece x on top), surrounded by four P-prisms.

One can also assemble a Dorian cube from five cuboids, using one 1 × 3 × 5, one 2 × 2 × 5, and three 2 × 3 × 5s. Indeed, there are zillions more ways, too many to count.

341. (a) Make an exact cover problem in which a and A, b and B, ... , f and F are required to be in symmetrical position; there are respectively (86, 112, 172, 112, 52, 26) placements for such 10-cubie “super-pieces.” Furthermore, the author decided to force piece m to be in the middle of the top wall. Solutions were found immediately! So piece x was placed in the exact center, as an additional desirable constraint. Then there were exactly 20 solutions; the one below has also n, o, and u in mirror-symmetrical locations.

(b) The super-pieces now have (59, 84, 120, 82, 42, 20) placements; the author also optimistically forced j, k, and m to be symmetrical about the diagonal, with m in the northwest corner. A long and apparently fruitless computation (34.3 teramems) ensued; but — hurrah — two closely related solutions were discovered at the last minute.

(c) This computation, due to Torsten Sillke [see Cubism For Fun 27 (1991), 15], goes much faster: The quarter-of-a-box shown here can be packed with seven non-x pentacubes in 55356 ways, found in 1.3 Gμ. As in answer 294, this yields a new exact cover problem, with 33412 different options.

Another 11.8 Gμ then yields seven suitable partitions into four sets of seven, one of which is illustrated below. [See also Cubism For Fun 49 (1999), 26.]

342. As in previous exercises, the key is to reduce the search space drastically, by asking for solutions of a special form. (Such solutions aren’t unlikely, because pentacubes are so versatile.) Here we can break the given shape into four pieces: Three modules of size 3^3+2^3 to be packed with seven pentacubes, and one of size 4^3 − 3 · 2^3 to be packed with eight pentacubes. The first problem has 13,587,963 solutions, found with 2.5 Tμ of computation; they involve 737,695 distinct sets of seven pentacubes. The larger problem has 15,840 solutions, found with 400 Mμ and involving 2075 sets of eight. Exactly covering those sets yields 1,132,127,589 suitable partitions; the first one found, {a, A, b, c, j, q, t, y}, {B, C, d, D, e, k, o}, {E, f, l, n, r, v, x}, {F, m, p, s, u, w, z}, works fine. (We need only one partition, so we needn’t have computed more than a thousand or so solutions to the smaller problem.)

Pentacubes galore: Since the early 1970s, Ekkehard Künzell and Sivy Farhi have independently published booklets that contain hundreds of solved pentacube problems.

343. We can use an instructive variety of methods to deduce that the tallest towers have heights (hO, hP,..., hZ) = (12, 29, 28, 28, 29, 25, 26, 23, 24, 17, 28, 27): Case O is trivial. A perfect tower for P was published by S. Farhi in Pentacubes, 5th edition (1981), Fig. 78. And it’s easy to show that hW ≤ 24, because r, t, v, x, z can’t be placed.

Factorization yields most of the upper bounds. For example, let the cells of a tower for R be {00k, 01k, 11k, 12k, 21k | 1 ≤ k ≤ h}, and add a new “weight” column to the exact-cover matrix, representing the sum of all items/columns 00k and 12k. (Thus the option ‘y 212 311 312 412 512’ has weight 4.) An exact cover by disjoint options/rows will then make the new column sum 2h. But the maximum weights of the pentacubes (a, A, ... , f, F, j, k, ... , z) are respectively (1, 1, 1, 1, 3, 3, 3, 3, 2, 2, 2, 2, 2, 2, 1, 1, 3, 5, 3, 4, 2, 3, 0, 3, 0, 0, 0, 4, 0). Their sum is 57; hence hR ≤ 57/2 < 29.

Similar arguments prove that hU < 27, hV < 24, hX < 18, hZ < 28. But case T is more complicated. Let’s introduce a column for the weights (100 · 00k) + (100 · 02k)+ (10 · 11k) + (101 · 21k), and compute the 29 maximum weights (312, 312, 310, 310, 311, 311, 221, 221, 210, 210, 220, 220, 220, 210, 211, 210, 310, 505, 323, 414, 300, 323, 400, 400, 400, 300, 200, 414, 400). The heaviest 27 sum to 8296, which is less than 311 · 27; hence hT < 27. And if hT = 26, further study shows that we must omit x and two of {e, E, k, m}. Moreover, each piece must use an option of maximum weight, except that c and C should use weight 310. These restrictions narrow down the search considerably; Algorithm X is able to prove that hT < 26 in 11 Tμ (and Algorithm M in 7.6 Tμ).

It’s difficult to prove that hQ < 29, and even harder to prove that hY < 29. But in both cases a suitable weighted factorization makes the calculations feasible. (See www.math.uni-bielefeld.de/~sillke/POLYCUBE/TOWER/pentacube.)

Such weights also greatly accelerate the successful searches, for towers of maximum height. Here are some that were hardest-to-find (add piece ‘s’ atop the first one):

344. Reduce the placements that occupy the center cell from 72 to 3. That problem has 2528 solutions, found by Algorithm X in 25 Gμ; and those solutions form 1264 mirror-symmetric pairs. [See C. J. Bouwkamp and D. A. Klarner, JRM 3 (1970), 10–26.]

345. A variation of even/odd coordinates works nicely: Let the pieces fill 13 cells like (x, y, z)+{(±1, ±1, ±3), (1, ±1, ±1)}, xyz odd, where the items (x, y, z) for 0 ≤ x, y ≤ 10 and 0 ≤ z ≤ 6 are primary for x, y, z even and secondary for x, y, z odd. The solution is unique. [This puzzle, marketed as “Vier Farben Block,” was designed by T. Geerinck in 2004.]

001122

001122

334444

334444

556677

556677

001122

888899

834894
a34a94
aaaa99

556677

001122

888899

83b89b
a3ba9b
aaaa99

556677

001122

001122

33bbbb

33nnnn

556677

556677

346. (a) Shifting by multiples of (0, 1, 1) gives N disjoint tripods whose corners are on layer 0 of the torus, filling all cells of that layer except for a (possibly broken) diagonal, and also filling all cells of such a diagonal on layer 1. We can plug the holes on layer 0 by appropriately placing N tripods whose corners are on layer N − 1. And so on.

(b) Here’s a way to pack twelve of them into a 3 × 6 × 6 torus. (Is 7/9 optimum?)

012600

112371

222348

933345

0a4445

01b555

066678

917778

9a2888

9ab399
aab64a
bbb675

0..6..

.1..7.

..2..8

9..3..

.a..4.

..b..5

(c) Place 13 tripods in a 6 × 6 × 6 torus, with corners at (0, 0, 0), (0, 1, 1), (0, 2, 2), (1, 1, 3), (1, 2, 4), (2, 3, 2), (2, 4, 4), (3, 3, 3), (3, 4, 5), (4, 4, 0), (4, 5, 1), (5, 0, 5), (5, 5, 3).

(d) One can place 2r(l, m, n) nonoverlapping tripods in a 2l × 2m × 2n torus, by putting the tripod corners at the positions of the pod corners, plus (0, 0, 0) and (l, m, n).

(e) With one primary item # and lmn secondary items xyz, and with options such as ‘# 123 023 103 113 120 121 122’ (one for each pod with 0 ≤ x < l, 0 ≤ y < m, 0 ≤ z < n), we can find solutions with t pods by giving multiplicity t to #. Furthermore we can save time by letting the items 000 and (l−1)(m−1)(n−1) be primary, because those two pods can be assumed to be present. In this way we find 444 ↦ 8, 445 ↦ 9, 446 ↦ 9, 455 ↦ 10, 456 ↦ 10, 466 ↦ 12, 555 ↦ 11, 556 ↦ 12, 566 ↦ 13, 666 ↦ 14. (Algorithm M can determine that r(6, 6, 6) < 15 in reasonable time, 253 Gμ, despite its rather weak heuristics for pruning the search. But the SAT solver Algorithm 7.2.2.2C solves this problem in only 2 Gμ; it can also establish that r(7, 7, 7) = 19 in 169 Gμ, while Algorithm M as it stands would be hopeless for that task.)

[Notes: Sherman Stein initiated the study of tripods (actually an n-dimensional generalization called “semicrosses”) in IEEE Trans. IT-30 (1984), 356–363; see also his paper with W. Hamaker on pages 364–368. They proved that the function r(n) = r(n, n, n) is Ω(n1.516), and that r(l, n, n)/n approaches a limit as n → ∞. The initial values (r(1),...,r(9)) = (1, 2, 5, 8, 11, 14, 19, 23, 28) were found by C. Morgan, in an undergraduate project at the University of Warwick in 2000; see also S. Szabό, Ann. Univ. Sci. Budapestinensis, Sect. Computatorica 41 (2013), 307–322. With extensive computations, P. R. J. Östergård and A. Pöllöanen have proved that r(10) = 32 and (surprisingly) that r(11) = 38 [Discrete and Computational Geometry 61 (2019), 271– 284]. See also A. Tiskin, Discrete Math. 307 (2007), 1973–1981, who showed among other things that r(12) ≥ 43, r(n) = Ω(n1.534), r(n) = O(n^2/(log n)1/15).]

347. Fourteen proofs have been given by S. Wagon, AMM 94, (1987), 601–617. [For generalizations, see R. J. Bower and T. S. Michael, Math. Magazine 79 (2006), 14–30.]

348. See F. W. Barnes’s complete solution, Discrete Mathematics 133 (1994), 55–78.

349. Let t = s/4. Each brick of an m-brick packing contains at least one of the 27 “special points” {(it, jt, kt) | 0 < i,j,k < 4}, because a, b, and c exceed t. Hence m ≤ 27.

In a packing with m = 27, each of the “special lines” l*jk, li*k, lij* with two coordinates fixed will be totally full, because the bricks collectively occupy 27(a + b + c) units of space on those lines. The special lines also intersect the bricks in 27 segments of each length a, b, c; hence each special line has a segment of each length.

Thus we’re led to solve an XCC problem with primary items pijk, l*jk, li*k, lij* and secondary items xijk, yijk, zijk, and with options like ‘pijk xijk:π_1 yijk:π_2 zijk:π_3’ and ‘li*k yi_1k:π_1 yi_2k:π_2 yi_3k:π_3’, where π_1π_2π_3 is a permutation of {a, b, c}. That problem has 7712 solutions, when we fix one of the six options for p_111.

Only 168 of those solutions, in 21 equivalence classes under the 48 symmetries of the cube, actually pack properly when (a, b, c) = (2, 3, 4). And it can be shown that those 21 solutions will solve Hoffman’s problem for arbitrary (a, b, c). Here, for example, is the unique solution that is “self-dual” — isomorphic to itself when a ↔ c:

[See Hoffman’s exposition in The Mathematical Gardner (1981), 212–225.]

350. Set this up for Algorithm M with 28 instances of a 3×4×5 brick and 48 instances of a single cubie. We can omit all options where a brick lies 1 or 2 units from a face but not on the face, because the brick could move outward in such solutions. We can also force the placement of a brick at corner (0, 0, 0). Furthermore, an empty corner would imply at least 27 cubies there; hence we can omit placing a cubie in any corner except (11, 11, 11). This problem, with 715 options of size 61 and 1721 options of size 2, has 112 solutions(!), found in 440 Gμ. (The author’s first attempt, in 2004, took much longer.)

There are three species of solutions: (i) Pack seven bricks into 5 × 7 × 12; arrange four of those in a pinwheel (see exercise 365), leaving a 2 × 2 × 12 hole. (ii) Pack 12 into 5 × 12 × 12; add a pinwheel of four 5 × 7 × 7s, each of which is a pinwheel of four 3 × 4 × 5s. (iii) Assemble the bricks in a bizarre way that includes two such 5 × 7 × 7s:

Types (i), (ii), (iii) contribute 6+10+4 nonisomorphic solutions. [George Miller’s puzzle with bricks of tricolored faces is called Perfect Packing, because 28 is a perfect number.]

351. (Generalizing exercise 349, Hoffman observed that such a construction would yield a nice geometrical way to prove the inequality (abcde)1/5 ≤ (a + b + c + d + e)/5.)

352. None. But any eleven of the “hypersolid pentominoes” can easily be squeezed in;
for example, is one way to pack all but V.

353. There are exactly 9 (including a mirror pair). They pack a 3 × 3 × 3 cube in 48 · 8789 ways, such as . [See J. Lou, Danish patent 126840 (1973).]

354. (a) Let cell (x, y) of a polyomino correspond to (–x, x, y, –y). Let cell (x, y) of a polyhex, as represented in exercise 315, correspond to (0,x,y, –x – y).

(b) A polysphere is planar if and only if the differences between its adjacent cells lie in a plane. Each of those differences has the form eij = ei –ej, where e_1 = (1, 0, 0, 0), ... , e_4 = (0, 0, 0, 1). Three such differences can’t be linearly independent yet lie in a plane; the linearly dependent cases are polyominoes and/or polyhexes.

(c) Every connected graph has at least one vertex whose removal doesn’t disconnect the graph. So the result follows by induction on n.

(d) An orthogonal matrix fixes w + x + y + z if and only if its row and column sums are 1. The matrices (i) T and (ii) R below respectively rotate by 120° about x = y = z and by 90° about (x = y) ∧ (w = z).

T=(1000001000010100);R=12(1−111111−1−111111−11);R2=(0001001001001000);H=16(5−1−13−1−153−15−13333−3).

(e) The matrices (i) R^2 and (ii) H above respectively rotate by 180° about (x = y) ∧ (w = z) and about (x = y) ∧ (w = 3z – 2x). Thus H can be used when z = 0.

(f) Suppose V′={v1′,...,vn′} is a rotation of V = {v_1,..., vn}⊂ S, where vk = (wk, xk, yk, zk), vk′T=(wk′,xk′,yk′,zk′)T=QvkT, v1=v1′=(0,0,0,0), and v_2 = e_12 = (1, –1, 0, 0). The matrix Q = (qij) is orthogonal, with row and column sums and determinant 1. By applying an even permutation to the coordinates of v′ and the rows of Q, we can assume without loss of generality that v2′=e12=v2. Hence qk_1 = qk_2 + δk_1 – δk_2, q_11 = q_22. If Q ≠ I we have vp−vq=eij≠ei′j′=vp′−vq′ for some p, q, i, j, i′, and j′, with i < j. By orthogonality, e_12 ⋅ eij = e_12 ⋅ ei′j′ ∈ {–1, 0, +1}.

If e_12 ⋅ eij = 1, there are six cases, depending on (i, j, i′,j′): (1, 3, 1, 4) implies Q = TH; (1, 4, 1, 3) implies Q = HT^2; (1, 3, 4, 2) implies Q = T^2RT or THR^3T 2; (1, 4, 3, 2) implies Q = T 2RT or HR; (1, 3, 3, 2) and (1, 4, 4, 2) are impossible.

If e_12 ⋅ eij = 0, we have (i, j, i ′,j ′) = (3, 4, 4, 3) and Q is forced to be TR. Finally, the case (i, j, i ′,j ′) for e_12 ⋅ eij = –1 is the same as the case (i, j, j ′,i ′) for e_12 ⋅ eij = +1.

Note: Some authors represent S as the set of integer triples (X, Y, Z) with X +Y +Z even. The Hadamard transform provides an isomorphism between these representations: If –2M is the upper left 4 × 4 submatrix of 7.2.1.1–(21), we have M^2 = I, det M = 1, and M takes (–x – y – z, x, y, z) ↦ (0,x + z, y + z, x + y) = (0,X,Y,Z).

355. (a) Normalize the given polysphere by subtracting (xmin ,ymin ,zmin), to get its base placement. Then, for each base placement P , form up to three others until no more can be formed: (i) Replace each xyz by yzx. (ii) Replace each xyz by (x + y + z)(t – z)(t – x), for some large t; then normalize. (iii) If z = 0 in each cell of P , replace each xy0 by yx0.

[The (X, Y, Z) representation mentioned in answer 354 suggests “polyjubes” — George Sicherman’s name for the sets of edge-connected cubes that don’t touch face-to-face. Transformation (iii) does not apply to polyjubes; hence there are 5 trijubes and 28 tetrajubes. Polyjubes are also equivalent to “polyrhons” — the connected sets of rhombic dodecahedra, which are the Voronoi regions of the face-centered cubic lattice. See S. Coffin, The Puzzling World of Polyhedral Dissection (1990), Figure 167.]

(b) Phenalene has eight base placements; in lexicographic order they are {000, 001, 010}, {000, 001, 100}, {000, 010, 100}, {001, 010, 011}, {001, 010, 100}, {001, 100, 101}, {010, 100, 110}, {011, 101, 110}. The straight trisphere has six base placements, namely {000, 001, 002}, {000, 010, 020}, {000, 100, 200}, {002, 011, 020}, {002, 101, 200}, {020, 110, 200}. The bent trisphere has twelve, from {001, 010, 101} to {011, 100, 110}. And phenanthrene has twenty-four, from {000, 001, 011} to {020, 101, 110}.

(c) There are 853 connected subsets, with 475 different base placements. (Each placement with max(x + y + z) = (1, 2, 3) occurs respectively (10, 4, 1) times.) They form 25 distinct tetraspheres — five from tetrominoes and six additional planar pieces from tetrahexes, plus four nonplanar nonchiral pieces and five chiral pairs:

Each piece has been given an identifying letter. This chart shows the number of base placements and the number of occurrences in simplex (3, 3, 3, 3, 3, 0, 0), as well as the lexicographically smallest base placement. Notice that j and p have 48 base placements, while a polycube can have at most 48. Piece s is simplex (1, 1, 1, 1, 1, 0, 0), a tetrahedron with four equidistant spheres. Piece x is perhaps the most fascinating to play with.

[The tetraspheres were first enumerated by K. Takizawa; then T. Sillke enumerated the nonplanar polyspheres of larger sizes. See B. Wiezorke, Cubism For Fun 25, part 3 (1990), 10–17; G. Bell, Cubism For Fun 81 (2010), 18–23; OEIS A038174.]

356. (a) The n-tetrahedron, which is the same as simplex (n – 1, n – 1, n – 1,n – 1, n–1, 0, 0), has base placement {xyz | x, y, z ≥ 0, x+y +z < n}; (n+23) cells. (It has one other base placement, namely {(n–1–x)(n–1–y)(n–1–z) | x, y, z ≥ 0, x+y+z < n}.)

One of the 12 base placements of the m × n roof is {x(y+k)(m–1–y) | k ≥ 0, 0 ≤ x < n – k, 0 ≤ y < m – k}. If m ≤ n, it has m(m + 1)(3n – m +1)/6 cells.

The stretched m × n roof is based on slicing the face-centered cubic lattice into layers with constant y –z. (Each cell has two neighbors on its own layer, four neighbors on each adjacent layer, and two neighbors that are two layers away.) One of its 12 base placements is {(x+m–1–y)(y+k)y | k ≥ 0, 0 ≤ x < n – k, 0 ≤ y < m – k}.

(b) Let’s call the four shapes T_4, R3×4, S3×4, and S4×3. Here are the stats:

Shape

Total

All planar

Mixed

All nonplanar

multisets(sets)

(balanced)

(balanced)

(chiral)

(balanced)

(chiral)

T_4

2952(1211)

174(34)

308(115)

2442(1062)

2(0)

26(0)

R3×4

11531(6274)

372(69)

1250(583)

9818(5608)

3(0)

88(14)

S3×4

1184(480)

51(6)

108(48)

1014(426)

1(0)

10(0)

S4×3

266(52)

2(0)

27(8)

234(44)

1(0)

2(0)

For example, {j, j, p, p, t} is one of 174 multisets of five planar pieces that can make T_4. [In fact, the solution is unique — and {j, j, p, p, t} also uniquely solves R3×4 and S3×4! G. Bell used this fact as the basis for his elegant Triple Pyradox puzzle; see Cubism For Fun 94 (2014), 10–13.] Of those 174 cases, 34 have five different pieces; for instance, {n, o, p, u, y} is one of only seven that contains y, the “propeller.”

Many other suitable sets of five mix planar pieces with nonplanar ones. Of these, 115 (like {g, G, i, s, x}) are closed under reflection; that one has 24 solutions, all essentially the same. The other 1062 form 531 mirror-image pairs (like {d, e, f, G, i} and {D, E, F, g, i}); every solution for a chiral set has 12 equivalents, not 24.

Algorithm M discovers all such solutions quickly, if we assign multiplicity [0 .. 5] to each piece. There are respectively (88927, 77783, 3440, 996) solutions to (T_4, R3×4, S3×4, S4×3), without symmetry removal; they’re found in (840, 607, 48, 13) Mμ.

Six of the multisets — three mirror pairs — are actually able to make all four shapes. These versatile combinations of pieces are {e, g, g, p, p} and {E, G, G, p, p}, {g, j, p, p, p} and {G, j, p, p, p}, {g, p, p, p, p} and {G, p, p, p, p}.

There’s an obvious, yet interesting, way to make T_4 with the “pure” multiset {s, s, s, s, s}. The only other pure multiset that works is {p, p, p, p, p}, which is able to form both T_4 and R3×4, as well as many other shapes noted by W. Schneider in 1995.

[A 2 × 7 roof also has 20 cells. So we might want to consider additional stats:

R2×7

3940(1628)

608(116)

1296(512)

1970(1000)

14(0)

52(0)

S2×7

426(84)

58(4)

48(20)

306(60)

2(0)

12(0)

S7×2

4(0)

0(0)

0(0)

0(0)

2(0)

2(0)

The long and skinny S7×2 can be made in only two ways, both with x in the middle, surrounded by g’s or G’s. The set {i, j, n, o, p} packs both S2×7 and S7×2, as well as T_4.]

(c) Let’s name the trispheres 1, 2, 3, 4, according to the squared distance between the two farthest-apart cells; thus the pieces in exercise 355 are 2, 4, 1, 3. The pyramid P_4 is buildable from 296 such multisets, many of which allow huge numbers of solutions. (For example, each of the ten multisets that contain {1, 1, 2, 2, 3, 3, 4, 4} leads to more than 30,000 solutions; {1, 1, 2, 2, 2, 3, 3, 4, 4, 4} has more than 120,000!) Most interesting are the cases with unique solution ({2, 2, 4, 4, 4, 4, 4, 4, 4, 4}‡, {1, 1, 1, 4, 4, 4, 4, 4, 4, 4}, {1, 2, 2, 2, 2, 2, 2, 2, 2, 2}), or with just two solutions ({2, 2, 2, 2, 2, 2, 2, 2, 2, 2}†, {1, 1, 3, 3, 3, 3, 3, 3, 3, 3}, {2, 4, 4, 4, 4, 4, 4, 4, 4, 4}‡); † = noted by L. Gordon (1986); ‡ = noted by J. Becker (2009). The stretched pyramid S_4 has 213 such multisets, all of which can also make P_4. Unique solutions occur for {1, 1, 1, 3, 4, 4, 4, 4, 4, 4} and {1, 3, 3, 3, 3, 3, 3, 3, 4, 4}; almost for {3, 3, 3, 3, 3, 3, 3, 3, 4, 4}.

Historical notes: The first polysphere puzzle may have been “Pyramystery,” copyright by Piet Hein in 1967 when his Soma cube was becoming popular. Pyramystery had the six pieces {1, 1, 3, 4, o, p}; Hein knew that it could form T_4, as well as two copies of T_3, and several planar designs. A similar puzzle of unknown origin, called Kugelpyramide, may have been created earlier, because it was seen by B. Wiezorke in 1968. Kugelpyramide’s pieces, {1, 3, 4, 4, o, p} were slightly different. With either Pyramystery or Kugelpyramide one can make T_4, T_3 + T_3, R3×4, R2×7, and S2×7; and with the not-thought-of pieces {1, 2, 3, 4, o, p}, one could have made also S3×4 but not T_3 + T_3. The first puzzle to mix polyomino-type polyspheres with polyhex-type polyspheres — a nonobvious possibility — was Tetra, by A. Kuwagaki and S. Takenaka; see Sugaku Seminar 11, 7 (July 1972), cover, 34–38; also U.S. Patent 3837652 (1974). That patent describes making P_3 from the dispheres and trispheres, and making the 44-ball octahedron P4P3R from the planar tetraspheres {i, j, l, n, o, p, q, t, u, y, z}. In those early days, the stretched roofs and pyramids weren’t known to be possible; they were first introduced by Leonard Gordon, in his WARP-30 puzzle (Kadon Enterprises, 1986).

(d) The unique base placement is {xyz | x, y, z ∈ {0, 1, 2, 3},x ≠ y ≠ z ≠ x}. Stats are 95(0) 5(0) 13(0) 70(0) 3(0) 4(0). Only pieces a, c, d, q, u will fit in this shape. Here’s how to make it with {a, a, c, d, u, u}, {c, c, c, C, C, C}, or {u, u, u, u, u, u}:

(Note that {q, q, q, q, q, q} is trivial.) This is a hollow object that can’t stand on its own.

357. Truncated octahedra are the Voronoi regions of the “body-centered cubic lattice,” which is less tight than the face-centered cubic lattice: It can be represented as the set of all integer triples (x, y, z) with x mod 2 = y mod 2 = z mod 2. Two truncated octahedra whose centers are two such points are adjacent if and only if the distance between those points is either 3 (eight neighbors, joined at hexagonal faces) or 2 (six neighbors, joined at square faces). There are 2 displatts, 6 trisplatts, and 44 tetrasplatts — including 9 chiral pairs. [See M. Owen and M. Richards, Eureka 47 (1987), 53–58.]

Base placements can be found almost as in exercise 324, except that we must set (x, y, z) ↦ (y, 2 ⌈xmax⌉ –x, z). Furthermore, each base placement should be normalized, by adding (±1, ±1, ±1) if needed, so that xmin + ymin + zmin ≤ 1.

[One might also consider truncating further, leaving only the union of four small hexagonal prisms between diametrically opposite hexagonal faces. This yields a sub-family of polysplatts called “polycrunches” — named and enumerated by G. Sicherman: Adjacent crunches, with centers 3 apart, are pasted together where the prisms meet. The polycrunch family has 1 monocrunch, 1 dicrunch, 3 tricrunches, and 14 tetra-crunches (including 2 chiral pairs). The tricrunches have respectively (4, 12, 12) base placements; the tetracrunches have respectively (4, 6, 6, 8, 12, 12, 12, 12, 24, ... , 24).]

358. This fascinating packing is considerably more difficult than the other. For example, there are six distinct trihexaspheres, having respective angles of (60°, 90°, arccos(–1/3) ≈ 109.5°, 120°, arccos(–5/6) ≈ 146.4°, 180°) and respective maximum squared distances (1, 2, 8/3, 3, 11/3, 4). G. Bell has discovered a convenient way to represent magnified polyhexaspheres within the face-centered cubic lattice: Consider the subset S^ of S whose elements have the special form αj+βk+γl for integers j, k, l, where α = (0, 3, –3, 0), β = (0, 0, 3, –3), γ_2l = (6l, –2l, –2l, –2l), and γ_2l+1 = (6l +3, –3 – 2l, –l, –2l). Two cells of S^ are called adjacent if the distance between them is 18. Thus each cell v of layer l has six neighbors v ± {(0, 3, –3, 0), (0, 0, 3, –3), (0, –3, 0, 3)} on the same level; three neighbors v + A[l even] + B[l odd] on level l + 1, where A = {(3, –3, 0, 0), (3, 0, –3, 0), (3, 0, 0, –3)} and B = {(3, 1, –2, –2), (3, –2, 1, –2), (3, –2, –2, 1)}; and three neighbors v – A[l odd] – B[l even] on level l – 1.

All of the tetraspheres are tetrahexaspheres, because they fit on at most two levels. But many of the pentaspheres, for example the planar one for pentomino T, are not pentahexaspheres. A polyomino polyhexasphere exists if and only if the polyomino fits in a 2×k box: Connected subsets of {(0, 0, 3k, –3k), (–3, –1, 2+3k, 2–3k)} are OK.

The matrices T and R^2HR^2 of answer 354 are rotations of S^. Therefore we can obtain equivalent base placements in the manner of answer 355, replacing each xyz by either yzx or (y+23w)(x+23w)(z+23w), where w = –x – y – z. Normalize a placement by adding or subtracting 666 or 33¯0 or 033¯ or 3¯03. But the analysis is still incomplete: Are further transformations of base placements needed? How many n-hexaspheres are possible, for n = 4, 5, ... ? [See Cubism For Fun 106 (2018), 24–29.]

359. First we realize that every edge of the square must touch at least three pieces; hence the pieces must in fact form a 3 × 3 arrangement. Consequently any correct placement would also lead to a placement for nine pieces of sizes (17 – k) × (20 – k), ... , (24 – k) × (25 – k), into a (65 – 3k) × (65 – 3k) box. Unfortunately, however, if we try, say, k = 16, Algorithm X quickly gives a contradiction.

But aha — a closer look shows that the pieces have rounded corners. Indeed, there’s just enough room for pieces to get close enough together so that, if they truly were rectangles, they’d make a 1 × 1 overlap at a corner.

So we can take k = 13 and make nine pieces of sizes 4 × 7, ... , 11 × 12, consisting of rectangles minus their corners. Those pieces can be packed into a 26×26 square, as if they were polyominoes (see exercise 266), but with the individual cells of the enclosing rectangle treated as secondary items because they needn’t be covered. (Well, the eight cells adjacent to corners can be primary.) We can save a factor of 8 by insisting that the 9 × 11 piece appear in the upper left quarter, with its long side horizontal.

Algorithm X solves that problem in 620 gigamems — but it finds 43 solutions, most of which are unusable, because the missing corners give too much flexibility. The unique correct solution is easily identified, because a 1 × 1 overlap between rectangles in one place must be compensated by a 1 × 1 empty cell between rectangles in another. The resulting cross pattern (like the X pentomino) occurs in just one of the 43.

360. Let there be mn primary items pij for 0 ≤ i < m and 0 ≤ j < n, one for each cell that should be covered exactly once. Also introduce m primary items xi for 0 ≤ i < m, as well as n primary items yj for 0 ≤ j < n. The exact cover problem has (m+12)⋅(n+12) options, one for each subrectangle [a..b) × [c..d) with 0 ≤ a < b ≤ m and 0 ≤ c < d ≤ n. The option for that subrectangle contains 2 + (b – a)(d – c) items, namely xa, yc, and pij for a ≤ i < b, c ≤ j < d. The solutions correspond to reduced decompositions when we insist that each xi be covered [1 ..n] times and that each yj be covered [1 ..m] times. (We can save a little time by omitting x_0 and y_0.)

The 3×5 problem has 20165 solutions, found in 18 Mμ. They include respectively (1071, 3816, 5940, 5266, 2874, 976, 199, 22, 1) cases with (7, 8, ... , 15) subrectangles.

[See C. J. Bloch, Environment and Planning B6 (1979), 155–190, for a complete catalog of all reduced decompositions into at most seven subrectangles.]

361. The minimum is m + n – 1. Proof (by induction): The result is obvious when m = 1 or n = 1. Otherwise, given a decomposition into t subrectangles, k ≥ 1 of them must be confined to the nth column. If two of those k are contiguous, we can combine them; the resulting dissection of order t – 1 reduces to either (m – 1) × n or m × n, hence t – 1 ≥ (m – 1) + n – 1. On the other hand if none of them are contiguous, the reduction of the first n – 1 columns is m × (n – 1); hence t ≥ m +(n – 1) – 1+ k.

Close examination of this proof shows that a reduced decomposition has minimum order t if and only if its boundary edges form m – 1 horizontal lines and n – 1 vertical lines that don’t cross each other. (In particular, the “tatami condition” is satisfied; see exercise 7.1.4–215.) See C. F. Earl, Environment and Planning B5 (1978), 179–187.

362. Simply remove the offending subrectangles, so that the cover problem has only ((m+12)−1)((n+12)−1) options. Now there are 13731 3×5 solutions, found in 11 Mμ, and (410, 1974, 3830, 3968, 2432, 900, 194, 22, 1) cases with (7, 8, ... , 15) subrectangles.

363. Introduce additional primary items Xi for 0 < i < m, to be covered [1 ..n – 1] times, as well as Yj for 0 < j < n, to be covered [1 ..m – 1] times. Then add items Xi for a < i < b and Yj for c < j < d to the constraint for subrectangle [a..b) × [c..d).

Now the 3 × 5 problem has just 216 solutions, found in 1.9 megamems. They include (66, 106, 44) instances with (7, 8, 9) subrectangles. Just two of the solutions are symmetric under left-right reflection, namely and its top-bottom reflection.

364. We can delete non-tromino options from the exact cover problem, thereby getting all faultfree tromino tilings that are reduced. If we also delete the constraints on xi and yj — and if we require Xi and Yj to be covered [1 ..n] and [1 ..m] times instead of [1 ..n – 1] and [1 ..m – 1] — we obtain all of the m × n faultfree tromino tilings.

It is known that such nontrivial tilings exist if and only if m, n ≥ 7 and mn is a multiple of 3. [See K. Scherer, JRM 13 (1980), 4–6; R. L. Graham, The Mathematical Gardner (1981), 120–126.] So we look at the smallest cases in order of mn: When (m, n) = (7, 9), (8, 9), (9, 9), (7, 12), (9, 10), we get respectively (32, 32), (48, 48), (16, 16), (706, 1026), (1080, 1336) solutions. Hence the assertion is false; a smallest counterexample is shown.

365. Augment the exact cover problem of answer 362 by introducing (m+12)+(n+12)−2 secondary items xab and ycd, for 0 ≤ a < b ≤ m and 0 ≤ c < d ≤ n, (a, b) ≠ (0,m), (c, d) ≠ (0,n). Include item xab and ycd in the option for subrectangle [a..b) × [c..d). Furthermore, cover xi [1 ..m – i] times, not [1 ..n]; cover yj [1 ..n – j] times.

366. The hint follows because [a..b) × [0 ..d) cannot coexist motleywise with its left-right reflection [a..b) × [n–d..n). Thus we can forbid half of the solutions.

Consider, for example, the case (m, n) = (7, 7). Every solution will include x_67 with some ycd. If it’s y_46, say, left-right reflection would produce an equivalent solution with y_13; therefore we disallow the option (a, b, c, d) = (6, 7, 4, 6). Similarly, we disallow (a, b, c, d) = (6, 7,c,d) whenever 7 – d < c.

Reflection doesn’t change the bottom-row rectangle when c+d = 7, so we haven’t broken all the symmetry. But we can complete the job by looking also at the top-row rectangle, namely the option where x_01 occurs with some yc d . Let’s introduce new secondary items t_1, t_2, t_3, and include tc in the option that has xc′ d′ with yc(7–c). Then we include t_1, t_2, and t_3 in the option that has x_01 with yc d′ for c ′ + d ′ > 7. We also add t_1 to the option with x_01 and y_25; and we add both t_1 and t_2 to the option with x_01 and y_34. This works beautifully, because no solution can have c = c ′ and d = d ′.

In general, we introduce new secondary items tc for 1 ≤ c < n/2, and we disallow all options x(m–1)m ycd for which c + d > n. We put tc into the option that contains x(m–1)m yc(n–c); t_1 thru t (n–1)/2 into the option that contains x_01 yc d when c ′ +d ′ > n; and t_1 thru tc –1 into the option that contains x_01 yc' (n–c′). (Think about it.)

For example, when m = n = 7 there now are 717 options instead of 729, 57 secondary items instead of 54. We now find 352546 solutions after only 13.2 gigamems of computation, instead of 705092 solutions after 26.4. The search tree now has just 7.8 meganodes instead of 15.7.

(It’s tempting to believe that the same idea will break top-bottom symmetry too. But that would be fallacious: Once we’ve fixed attention on the bottommost row while breaking left-right symmetry, we’ve lost all symmetry between top and bottom.)

367. From any m × n dissection of order t we get two (m+2) × (n+2) dissections of order t + 4, by enclosing it within two 1 × (m+1) tiles and two 1 × (n+1) tiles. So the claim follows by induction and the examples in exercise 365, together with a 5 × 6 example of order 10 — of which there are 8 symmetrical instances such as the one shown here. (This construction is faultfree, and it’s also “tight”: The order of every m × n dissection is at least m + n – 1, by exercise 361.)

In general, Helmut Postl observes that we can create nested motley dissections by motley-dissecting any subrectangle of a motley dissection (taking care not to repeat any internal boundary coordinates) and reducing the result. For example, one of the 2 ⋅ (6+3+3+3+1+9+3) = 56 ways to nest a pinwheel within the second motley 4 × 4 is shown here.

368. The number of subrectangles [a..b) × [c..d) that have either c = k or d = k, given k, is ≥ 2 when k ∈ {0,n} and ≥ 3 when 0 < k < n. Hence 2t ≥ 2+3(n – 1)+2.

369. All 214 of the 5×7 motley dissections have order 11, which is far short of (62)−1=14; and there are no 5×8s, 5×9s, or 5×10s. Surprisingly, however, 424 of the 696 dissections of size 6×12 do have the optimum order 20, and 7×17 dissections with the optimum order 27 also exist. Examples of these remarkable patterns are shown. (The case m = 7 is still not fully explored except for small n. For example, the total number of motley 7×17 dissections is unknown. No 7×18s exist, by exercise 368. If we restrict attention to symmetrical dissections, the maximum orders for 5 ≤ m ≤ 8 are 11 (5×7); 19 (6×11); 25 (7×15); 33 (8×21).)

370. The basic idea is to combine complementary options into a single option whenever possible. More precisely: (i) If a + b = m and c + d = n, we retain the option as usual; it is self-complementary. (ii) Otherwise, if a + b = m or c + d = n, reject the option; merging would be non-motley. (iii) Otherwise, if a + b > m, reject the option; we’ve already considered its complement. (iv) Otherwise, if b = 1 and c + d < n, reject the option; its complement is illegal. (v) Otherwise, if b > m/2 and c < n/2 and d > n/2, reject the option; it intersects its complement. (vi) Otherwise merge the option with its complement. For example, when (m, n) = (4, 5), case (i) arises when (a, b, c, d) = (1, 3, 2, 3); the option is ‘x_1 y_2 p_12 p_22 x_13 y_23’ as in answer 366. Case (ii) arises when (a, b, c, d) = (1, 3, 0, 1). Case (iii) arises when (a, b) = (2, 3). Case (iv) arises when (a, b, c, d) = (0, 1, 0, 1); the complement (3, 4, 4, 5) isn’t a valid subrectangle in answer 366. Case (v) arises when (a, b, c, d) = (1, 3, 1, 3); cells p_22 and p_23 occur also in the complement (1, 3, 2, 4). And case (vi) arises when (a, b, c, d) = (0, 1, 4, 5); the merged option is the union of ‘x_0 y_4 p_04 x_01 y_45 t_1 t_2’ and ‘x_3 y_0 p_30 x_34 y_01’. (Well, x_0 and y_0 are actually omitted, as suggested in answer 360.)

Size 8 × 16 has (6703, 1984, 10132, 1621, 47) solutions, of orders (26, ... , 30).

371. (a) Again we merge compatible options, as in answer 370. But now (a, b, c, d) → (c, d, n – b, n – a) → (n – b, n – c, n – b, n – a) → (n – b, n – a, c, d), so we typically must merge four options instead of two. The rules are: Reject if a = n – 1 and c + d > n, or c = n – 1 and a + b < n, or b = 1 and c + d < n, or d = 1 and a + b > n. Also reject if (a, b, c, d) is lexicographically greater than any of its three successors. But accept, without merging, if (a, b, c, d) = (c, d, n–b, n–a). Otherwise reject if b > c and b+d > n, or if b > n/2 and c < n/2 and d > n/2, because of intersection. Also reject if a + b = n or c + d = n, because of the motley condition. Otherwise merge four options into one.

For example, the merged option when n = 4 and (a, b, c, d) = (0, 1, 2, 4) is ‘x_0 y_2 p_02 p_03 x_01 y_24 t_1 x_2 y_3 p_23 p_33 x_24 y_34 x_3 y_0 p_30 p_31 x_34 y_24 p_00 p_10 x_02 y_01’, except that x_0 and y_0 are omitted. Notice that it’s important not to include an item xi or yj twice, when merging in cases that have a = c or b = d or a = n – d or b = n – c.

(b) With bidiagonal symmetry it’s possible to have (a, b, c, d) = (c, d, a, b) but (a, b, c, d) ≠ (n – d, n – c, n – b, n – a), or vice versa. Thus we’ll sometimes merge two options, we’ll sometimes merge four, and we’ll sometimes accept without merging. In detail: Reject if a = n – 1 and c + d > n, or c = n – 1 and a + b > n, or b = 1 and c + d < n, or d = 1 and a + b < n. Also reject if (a, b, c, d) is lexicographically greater than any of its three successors. But accept, without merging, if a = c = n – d = n – b. Otherwise reject if b > c or b > n – d or a + b = n or c + d = n. Otherwise merge two or four distinct options into one.

Examples when n = 4 are: ‘x_1 y_1 p_11 p_12 p_21 p_22 x_13 y_13’; ‘x_0 y_3 p_03 x_01 y_34 t_1 x_3 y_0 p_30 x_34 y_01’; ‘x_0 y_2 p_02 x_34 y_23 t_1 x_1 y_3 p_13 x_12 y_34 x_3 y_1 p_31 x_34 y_12 x_2 y_0 p_20 x_23 y_01’; again with x_0 and y_0 suppressed.

(c) The unique solution for n = 10 is shown. [The total number of such patterns for n = (10, 11, ... , 16) turns out to be (1, 0, 3, 6, 28, 20, 354). All 354 of the 16×16 solutions are found in only 560 megamems; they have orders 34, 36, and 38–44. Furthermore the number of n × n motley dissections with symmetry (a), for n = (3, 4, 5, ... , 16), turns out to be (1, 0, 2, 2, 8, 18, 66, 220, 1024, 4178, 21890, 102351, 598756, 3275503), respectively. Algorithm M needs 3.3 teramems when n = 16; those patterns have orders 4k and 4k + 1 for k = 8, 9, ... , 13.]

372. (a) This fact, and the others noted below, can be proved by induction on the number of rooms: If the lower right corner of the upper left room is a ⊥ junction, we can “flatten” and remove that room by bringing its right bound left; otherwise we can bring its bottom bound up. All floorplans can be built up by reversing this flattening process.

Let the rooms be r_1 ...rn in diagonal order and rp_1 ...rpn in antidiagonal order (left to right). Then ri ⇓ rj ⇔ i < j and i follows j in the permutation p = p_1 ...pn; ri ⇒ rj ⇔ i < j and i precedes j in p. The number of horizontal bounds is the number of descents in p, plus 2. The number of vertical bounds is the number of ascents, plus 2.

(b) Here’s the twin tree structure for the example. Notice that its leftward and rightward chains are the ordered sequences of rooms adjacent to the bounds.

Every twin tree structure arises in a remarkably simple way: Let p = p_1p_2 ...pn be any permutation of {1, 2,...,n}. Obtain T_0 by inserting p_1, p_2, ... , pn into an initially empty binary tree; obtain T_1 similarly by inserting pn, ... , p_2, p_1. Those trees can be constructed in linear time (exercise 6.2.2–50); and it’s easy to see that they are twins, both with inorder 12 ...n. Although different permutations can yield the same twin tree, exactly one Baxter permutation (exercise MPR–135) does so; and it can be computed from the twin tree in linear time(!). Thus there are nice one-to-one correspondences between floorplans, twin trees, and Baxter permutations.

[Floorplans are important in VLSI layout, where rooms correspond to modules and bounds correspond to channels. Twin trees were introduced by S. Dulucq and O. Guibert in Discrete Math. 157 (1996), 91–106, purely for their combinatorial interest, then applied to floorplans by B. Yao, H. Chen, C.-K. Cheng, and R. Graham in ACM Trans. Design Aut. Electronic Syst. 8 (2003), 55–80. See also J. M. Hart, Int. J. Comp. Inf. Sciences 9 (1980), 307–321; H. Murata, K. Fujiyoshi, T. Watanabe, and Y. Kajitani, Proc. Asia South Pacific Design Aut. Conf. 2 (1997), 625–633; E. Ackerman, G. Barequet, and R. Y. Pinter, Discrete Applied Mathematics 154 (2006), 1674–1684; and the author’s programs FLOORPLAN-TO-TWINTREE, TWINTREE-TO-BAXTER, BAXTER-TO-FLOORPLAN, available online (2021).]

373. The reduction of a perfectly decomposed rectangle is a motley dissection. Thus we can find all perfectly decomposed rectangles by “unreducing” all motley dissections.

For example, the only motley dissection of order 5 is the 3 × 3 pinwheel. Thus the perfectly decomposed m × n rectangles of order 5 with integer dimensions are the positive integer solutions to x_1 + x_2 + x_3 = m, y_1 + y_2 + y_3 = n such that the ten values x_1, x_2, x_3, x_1 + x_2, x_2 + x_3, y_1, y_2, y_3, y_1 + y_2, y_2 + y_3 are distinct. Those equations are readily factored into two easy backtrack problems, one for m and one for n, each producing a list of five-element sets {x_1,x_2,x_3,x_1 + x_2,x_2 + x_3}; then we search for all pairs of disjoint solutions to the two subproblems. In this way we quickly see that the equations have just two essentially different solutions when m = n = 11, namely (x_1,x_2,x_3) = (1, 7, 3) and (y_1,y_2,y_3) = (2, 4, 5) or (5, 4, 2). The smallest perfectly decomposed squares of order 5 therefore have size 11 × 11, and there are two of them (shown below); they were discovered by M. van Hertog, who reported them to Martin Gardner in May 1979. (Incidentally, a 12×12 square can also be perfectly decomposed.)

There are no solutions of order 6. Those of orders 7, 8, 9, 10 must come respectively from motley dissections of sizes 4 × 4, 4 × 5, 5 × 5, and 5 × 6. By looking at them all, we find that the smallest n × n squares respectively have n = 18, 21, 24, and 28. Each of the order-t solutions shown here uses rectangles of dimensions {1, 2,..., 2t}, except in the case t = 9: There’s a unique perfectly decomposed 24 × 24 square of order 9, and it uses the dimensions {1, 2,..., 17, 19}.

[W. H. Cutler introduced perfectly decomposed rectangles in JRM 12 (1979), 104–111.]

374. (a) False (but close). Let the individual dimensions be z_1, ... , z_2t, where z_1 ≤ ... ≤ z2t. Then we have {w_1,h_1} = {z_1,z_2t}, {w_2,h_2} = {z_2,z_2t–1}, ... , {wt,ht} = {zt,zt+1}; consequently z_1 < ... < zt ≤ zt+1 < ... < z_2t. But zt = zt+1 is possible.

(b) False (but close). If the reduced rectangle is m × n, one of its subrectangles might be 1 × n or m × 1; a motley dissection must be strict.

(c) True. Label the rectangles {a, b, c, d, e} as shown. Then there’s a contradiction: wb > wd ⇔ we > wc ⇔ he < hc ⇔ hd < hb ⇔ wb < wd.

(d) The order can’t be 6, because the reduction would then have to be a pinwheel together with a 1 × 3 subrectangle, and the argument in (c) would still apply. Thus the order must be 7, and we must show that the second dissection of exercise 365 doesn’t work. Labeling its regions {a,...,g} as shown, we have hd > ha; hence wa > wd. Also he > hb; so wb > we. Oops: wf > wg and hf > hg.

In the other motley 4 × 4 dissection of exercise 365 we obviously have
w_4 < w_5, w_4 < w_6, w_6 < w_7, h_4 < h_3, h_3 < h_1, h_4 < h_2;
therefore h_4 > h_5, h_4 > h_6, h_6 > h_7, w_4 > w_3, w_3 > w_1, w_4 > w_2. Now h_5 < h_6 ⇔ w_5 > w_6 ⇔ w_2 > w_3 ⇔ h_2 < h_3 ⇔ h_6 + h_7 < h_5. Hence h_5 < h_6 implies h_5 > h_6; we must have h_5 > h_6, thus also h_2 > h_3. Finally h_2 < h_1, because h_7 < h_5.

(e) The condition is clearly necessary. Conversely, given any such pair of solutions, the rectangles w_1 × αh_1, ... , wt × αht are incomparable for all large enough α.

[Many questions remain unanswered: Is it NP-hard to determine whether or not a given motley dissection supports an incomparable dissection? Is there a motley dissection that supports incomparable dissections having two different permutation labels? Can a symmetric motley dissection ever support an incomparable dissection?]

375. (a) By exercise 374(d), the widths and heights must satisfy

w_5 = w_2 + w_4,

w_6 = w_3 + w_4,

w_7 = w_1 + w_3 + w_4;

h_3 = h_4 + h_5,

h_2 = h_4 + h_6, + h_7

h_1 = h_4 + h_5 + h_6.

To prove the hint, consider answer 374(a). Each zj for 1 ≤ j ≤ t can be either h or w; then z_2t+1–j is the opposite. So there are 2t ways to shuffle the h’s and w’s together.

For example, suppose all the h’s come first, namely h_7 < ... <h_1 ≤ w_1 < ... <w_7:

1 ≤ h_7, h_7 +1 ≤ h_6, h_6 +1 ≤ h_5, h_5 +1 ≤ h_4, h_4 +1 ≤ h_4 + h_5, h_4 + h_5 +1 ≤ h_4 + h_6 + h_7, h_4 + h_6 + h_7 +1 ≤ h_4 + h_5 + h_6, h_4 + h_5 + h_6 ≤ w_1, w_1 +1 ≤ w_2, w_2 +1 ≤ w_3, w_3 +1 ≤ w_4, w_4 +1 ≤ w_2 + w_4, w_2 + w_4 +1 ≤ w_3 + w_4, w_3 + w_4 +1 ≤ w_1 + w_3 + w_4.

The least semiperimeter in this case is the smallest value of w_1 + w_2 + w_3 + w_4 + h_7 + h_6 + h_5 + h_4, subject to those inequalities; and one easily sees that the minimum is 68, achieved when h_7 = 2, h_6 = 3, h_5 = 4, h_4 = 5, w_1 = 12, w_2 = 13, w_3 = 14, w_4 = 15.

Consider also the alternating case, w_1 < h_7 < w_2 < h_6 < w_3 < h_5 < w_4 ≤ h_4 < w_2+w_4 < h_4+h_5 < w_3+w_4 < h_4+h_6+h_7 < w_1+w_3+w_4 < h_4+h_5+h_6. This case turns out to be infeasible. (Indeed, any case with h_6 < w_3 < h_5 requires h_4 + h_5 < w_3 + w_4, hence it needs h_4 < w_4.) Only 52 of the 128 cases are actually feasible.

Each of the 128 subproblems is a classic example of linear programming, and a decent LP solver will resolve it almost instantly. The minimum semiperimeter with seven subrectangles is 35, obtained uniquely in the case w_1 < w_2 < w_3 < h_7 < h_6 < h_5 < h_4 ≤ w_4 < w_5 < w_6 < w_7 < h_3 < h_2 < h_1 (or the same case with w_4 ↔ h_4) by setting w_1 = 1, w_2 = 2, w_3 = 3, h_7 = 4, h_6 = 5, h_5 = 6, h_4 = w_4 = 7. The next-best case has semiperimeter 43. In one case the best-achievable semiperimeter is 103!

To find the smallest square, we simply add the constraint w_1+w_2+w_3 +w_4 = h_7+ h_6 + h_5 + h_4 to each subproblem. Now only four of the 128 are feasible. The minimum side, 34, occurs uniquely when (w_1,w_2,w_3,w_4,h_7,h_6,h_5,h_4) = (3, 7, 10, 14, 6, 8, 9, 11).

(b) With eight subrectangles the reduced pattern is 4 × 5. We can place a 4 × 1 column at the right of either the 4 × 4 pattern or its transpose; or we can use one of the first two 4 × 5 patterns in exercise 365. (The other six patterns can be ruled out, using arguments similar to those of answer 374.) The labeled diagrams are

For each of these four choices there are 256 easy subproblems to consider. The best semiperimeters are respectively (44, 44, 44, 56); the best square sizes are respectively — and surprisingly — (27, 36, 35, 35). [With eight subrectangles we can dissect a significantly smaller square than we can with seven! Furthermore, no smaller square can be incomparably dissected, integerwise, because nine subrectangles would be too many.] One way to achieve 44 is with (w_1,w_2,w_3,w_4,w_5,h_8,h_7,h_6,h_5) = (4, 5, 6, 7, 8, 1, 2, 3, 8) in the third diagram. The only way to achieve a square of side 27 is with (w_1,w_2,w_3,w_4,w_5,h_8,h_7,h_6,h_5) = (1, 3, 5, 7, 11, 4, 6, 8, 9) in the first diagram.

These linear programs usually have integer solutions; but sometimes they don’t. For example, the optimum for the second diagram in the case h_8 < h_7 < w_1 < h_6 < w_2 < w_3 < w_4 < h_5 turns out to be 97/2, achievable when (w_1,w_2,w_3,w_4,w_5,h_8, h_7,h_6,h_5) = (7, 11, 13, 15, 17, 3, 5, 9, 17)/2. The minimum rises to 52, if we restrict to integer solutions, achieved by (w_1,w_2,w_3,w_4,w_5,h_8,h_7,h_6,h_5) = (4, 6, 7, 8, 9, 1, 3, 5, 9).

[The theory of incomparable dissections was developed by A. C. C. Yao, E. M. Reingold, and B. Sands in JRM 8 (1976), 112–119. For generalizations to three dimensions, see C. H. Jepsen, Mathematics Magazine 59 (1986), 283–292.]

376. This is an incomparable dissection in which exercise 374(d) applies. Let’s try first to solve the equations a(x+y+z) = bx = c(w+x) = d(w+x+y) = (a+b)w = (b+c)y = (b+c+d)z = 1, by setting b = x = 1. We find successively c = 1/(w+1), a = (1–w)/w, y = (w +1)/(w + 2), d = (w +2)/((w + 1)(w + 3)), z = (w + 1)(w +3)/((w + 2)(w + 4)). Therefore x+y +z –1/a = (2w +3)(2w^2+6w –5)/((w –1)(w +2)(w +4)), and we must have 2w^2+6w = 5. The positive root of this quadratic is w = (√ –3)/2, where √ = 19.

Having decomposed the rectangle (a+b+c+d)×(w+x+y+z) into seven different rectangles of area 1, we normalize it, dividing (a, b, c, d) by a+b+c+d=715(√+1) and dividing (w, x, y, z)by w+x+y+z=56(√−1). This gives the desired tiling (shown), with rectangles of dimensions 114(7−√)×115(7+√),542(−1+√)×115(1+√),521×35, 121(8−√)×115(8+√),121(8+√)×115(8−√), 542(1+√)×115(−1+√),114(7+√)×115(7−√).

[See W. A. A. Nuij, AMM 81 (1974), 665–666. To get eight different rectangles of area 1/8, we can shrink one dimension by 7/8 and attach a rectangle (1/8) × 1. Then to get nine of area 1/9, we can shrink the other dimension by 8/9 and attach a (1/9) × 1 sliver. And so on. The eight-rectangle problem also has two other solutions, supported by the third and fourth 4 × 5 patterns in exercise 375(b).]

377. (a) We can obtain h × w except when w is odd and h is not a multiple of 3. For if w is even, we can concatenate w/2 instances of size h × 2; if h is a multiple of 3, we can concatenate h/3 instances of size 3 × w; otherwise we can’t use concatenation to obtain w as the sum of two even numbers, or h as the sum of two multiples of 3.

(b) The shapes 2×3, 2×4, 2×5, 3×4, 3×5, 3×6, 3×7 are necessary and sufficient. (And then Λ(S) = { h×w | h > 1,w > 3}∪{ 2h×3 | h ≥ 1}.)

(c) S = {2×4, 3×8, 4×2, 8×3}.

(d) h×w ∈ S if and only if h = an′ for some a with ⌊m/n′⌋ < a < 2⌊m/n′⌋ + 2 and w = bn" for some b with ⌊m/n"⌋ < b < 2 ⌊m/n"⌋ + 2, where n′ = n/gcd(n, w) and n" = n/gcd(n, h).

378. Consider first a one-dimensional analog: If A is a set of positive integers, let Λ(A) be the integers obtainable by adding together one or more elements of A. We can prove that any set B of positive integers has a finite subset A such that B ⊆ Λ(A). For if B is empty, there’s nothing to prove; otherwise let b = min(B). Let qr be the smallest element of B such that qr mod b = r, for 0 ≤ r < b, or let qr be undefined if no such element exists. Then every element of B is some qr plus a multiple of q_0 = b.

Therefore in two dimensions, there’s a finite set X = {h_1×w_1,...,ht×wt} ⊆ T such that the width of every element of T is in Λ(X*), where X* = {w_1,...,wt} is the set of widths in X. Let p = h_1 ...ht be the product of all heights in X. It follows that p×w ∈ Λ(X) whenever h×w ∈ T .

For 0 ≤ r < p, let Tr be the elements h×w of T with h mod p = r, and let Qr be a finite subset of Tr such that every element of Tr has a width in Λ(Qr*). Let q be the largest height of any element of any Qr. Notice that if h×w ∈ T with h > q, and if h ′×w ′ ∈ Qh mod p, we have h×w ′ ∈ Λ(X ∪ Qr), because p×w ′ ∈ Λ(X) and h – h ′ is a positive multiple of p. Hence h×w ∈ Λ ({h×w ′ | h ′×w ′ ∈ Qr}) ⊆ Λ(X ∪ Qr).

Finally, for 1 ≤ i ≤ q, let Ti′ be the elements h×w of T with h = i, and let Pi be a finite subset of Ti′ such that every element of Ti′ has a width in Λ(Pr*). Then every element of T belongs to Λ(X ∪ Q_0 ∪ … ∪ Qp–1 ∪ P_1 ∪…∪ Pq).

[This argument extends to any number of dimensions. See N. G. de Bruijn and D. A. Klarner, Philips Research Reports 30 (1975), 337*–343*; Michael Reid, J. Combinatorial Theory A111 (2005), 89–105.]

379. A 2×5 packing is obvious; thus the basis contains 2×5 (and 5×2). The case 5×w and w > 2 has a packing only if 5×(w – 2) does. The case h = 3 is clearly impossible.

The case h = 7 is more interesting: 7×10 follows by concatenation, while 7×15 has 80 distinct and easily found solutions. Hence the basis contains 7×15 and 15×7.

This basis is complete: We’ve shown that if h is not a multiple of 5, h×w is possible whenever w is a multiple of 5, except when h = 1 or h = 3 or (h is odd and w = 5). If h and w are both multiples of 5, h×w is possible except when h or w equals 5 and the other is odd. [See W. R. Marshall, J. Combinatorial Theory A77 (1997), 181–192; M. Reid, J. Combinatorial Theory A80 (1997), 106–123.]

380. The minimum basis consists of 15×15 (see Fig. 73) plus 39 pairs {h×w, w×h}, where (h, w) ∈ {(5, 10), (9, 20), (9, 30), (9, 45), (9, 55), (10, 14), (10, 16), (10, 23), (10, 27), (11, 20), (11, 30), (11, 35), (11, 45), (12, 50), (12, 55), (12, 60), (12, 65), (12, 70), (12, 75), (12, 80), (12, 85), (12, 90), (12, 95), (13, 20), (13, 30), (13, 35), (13, 45), (14, 15), (15, 16), (15, 17), (15, 19), (15, 21), (15, 22), (15, 23), (17, 20), (17, 25), (18, 25), (18, 35), (22, 25)}. (This problem has a long history, going back to the discovery by David Klarner that ten one-sided Y pentominoes can be packed uniquely in a 5 × 10 box [Fibonacci Quarterly 3 (1965), 20]. Klarner eventually found 14 of the 39 basic pairs by hand, including the difficult case (12, 80). The other nine cases (12,w) were found by J. Bitner [JRM 7 (1974), 276–278], using a frontier-transition method that works much faster than Algorithm X in cases where h is much less than w. The complete set was nailed down by T. Sillke in 1992 [unpublished], then independently by J. Fogel, M. Goldenberg, and A. Liu [Mathematics and Informatics Quarterly 11 (2001), 133–137].)

381. Algorithm X quickly finds examples for n = 7, 11, 12, 13, 15, 16, 17; hence it’s possible for all n ≥ 11. [J. B. Kelly discovered the case n = 7 in AMM 73 (1966), 468. Are all packable rectangles consequences of this basis?]

382. Let the back corner in the illustration be the point 777, and write just ‘abcdef’ instead of [a..b)×[c..d)×[e..f). The subcuboids are 670517 (270601) 176705 (012706) 051767 (060127), 561547 (260312) 475615 (122603) 154756 (031226), 351446 (361324) 463514 (243613) 144635 (132436), 575757 (020202), 454545 (232323) — with the 11 mirror images in parentheses — plus the central cubie 343434. Notice that each of the 28 possible intervals is used in each dimension, except [0 ..4), [1 ..6), [2 ..5), [3 ..7), [0 ..7).

I started from a central cube and built outwards, all the while staring at the 24-cell in Hilbert’s Geometry and the Imagination.

— SCOTT KIM, letter to Martin Gardner (December 1975)

383. (Solution by Helmut Postl.) We can use the 7-tuples (2, 10, 27, 17, 11, 20, 5), (1, 14, 18, 8, 21, 24, 6), (3, 19, 16, 7, 34, 9, 4) to “unreduce” the 1st, 2nd, 3rd coordinates. For example, subcuboid 670517 becomes 5×(1+14+18+8+21)×(19+16+7+34+9+4). The resulting dissection, of a 92×92×92 cube into blocks of sizes 1×70×87, 2×77×88, 3 × 80 × 86, 4 × 67 × 91, 5 × 62 × 89, 6 × 79 × 90, 7 × 8 × 17, 9 × 51 × 65, 10 × 38 × 71, 11 × 21 × 34, 12 × 15 × 22, 13 × 25 × 30, 14 × 39 × 66, 16 × 18 × 27, 19 × 33 × 75, 20 × 47 × 61, 23 × 32 × 48, 24 × 36 × 76, 26 × 37 × 50, 28 × 40 × 43, 29 × 31 × 42, 35 × 44 × 53, 41 × 45 × 54, makes a fiendishly difficult puzzle.

How were those magic 7-tuples discovered? An exhaustive search such as that of exercise 374 was out of the question. Postl first looked for 7-tuples that led to very few dimensions in the “popular” ranges [13 .. 23] and [29 .. 39]. With luck, a large set of other 7-tuples would lead to no conflict in the 23 relevant subtotals; and with further luck, some of those wouldn’t conflict with each other.

(Postl also proved that no 91 × 91 × 91 decomposition is possible.)

384. The exact cover problem of answer 365 is readily extended to 3D: The option for every admissible subcuboid [a..b) × [c..d) × [e..f) has 6 + (b – a)(d – c)(f – e) items, namely xa yc ze xab ycd zef and the cells pijk that are covered.

We can do somewhat better, as in exercise 366: Most of the improvement in that answer can be achieved also 3Dwise, if we simply omit cases where a = l – 1 and either c + d > m or e + f > n. Furthermore, if m = n we can omit cases with (e, f) < (c, d).

Without those omissions, Algorithm M handles the case l = m = n = 7 in 98 teramems, producing 2432 solutions. With them, the running time is reduced to 43 teramems, and 397 solutions are found.

(The 7 × 7 × 7 problem can be factored into subproblems, based on the patterns that appear on the cube’s six visible faces. These patterns reduce to 5 × 5 pinwheels, and it takes only about 40 Mμ to discover all 152 possibilities. Furthermore, those possibilities reduce to only 5 cases, under the 48 symmetries of a cube. Each of those cases can then be solved by embedding the 5 × 5 reduced patterns into 7 × 7 unreduced patterns, considering 15^3 = 3375 possibilities for the three faces adjacent to vertex 000. Most of those possibilities are immediately ruled out. Hence each of the five cases can be solved by Algorithm C in about 70 Gμ — making the total running time about 350 Gμ. However, this 120-fold increase in speed cost the author two man-days of work.)

All three methods showed that, up to isomorphism, exactly 56 distinct motley cubes of size 7 × 7 × 7 are possible. Each of those 56 dissections has exactly 23 cuboids. Nine of them are symmetric under the mapping xyz ↦ (7 – x)(7 – y)(7 – z); and one of those nine, namely the one in exercise 382, has six automorphisms.

[These runs confirm and slightly extend the work of W. H. Cutler in JRM 12 (1979), 104–111. His computer program found exactly 56 distinct possibilities, when restricting the search to solutions that have exactly 23 cuboids.]

385. No; there are infinitely many. For example, Postl has constructed a primitive 11 × 11 × 13 by pasting Kim’s 7 × 7 × 7 to its mirror image, perturbing a few planes normal to the splice, and reducing.

386. The twelve possible symmetries can be represented as the permutations of {0, 1, 2, 3, 4, 5} defined by x ↦ (ax + b) mod 6, where a = ±1 and 0 ≤ b < 6; let’s denote that permutation by b or b, according to the sign of a. There are ten symmetry classes, depending on the automorphisms that are present: (i) all twelve; (ii) {0,0¯,2,2¯,4,4¯}; (iii) {0,1¯,2,3¯,4,5¯}; (iv) {0, 1, 2, 3, 4, 5}; (v) {0,3,0¯,3¯} or {0,3,2¯,5¯} or {0,3,4¯,1¯}; (vi) {0, 2, 4}; (vii) {0, 3}; (viii) {0,0¯} or {0,2¯} or {0,4¯}; (ix) {0,1¯} or {0,3¯} or {0,5¯}; (x) {0}.

(Types (ii), (iii) and (viii), (ix) depend on whether a reflection is left-right or top-down. Notice that there are 12/k base placements when there are k automorphisms.)

387. The 24 potential symmetries S can be represented as signed permutations of {±1,±2,±3}, meaning that coordinates are permuted and/or complemented. Using the notation of answer 7.2.1.2–20, they are 123,123¯,1¯23¯,...,321¯, where the number of inversions of the permutation plus the number of complementations is even.

Each of those symmetries is a rotation in 3-space about some line through the origin. (After a polycube has been rotated by one of its symmetries, we should shift the result, if necessary, to bring it into the original position.) For example, 1¯32 takes (x, y, z) ↦ (c – x, z, y); it’s a rotation of 180° about the diagonal line x = c/2, y = z. It’s a symmetry of the bent tricube {000, 001, 010} when c = 0; it’s a symmetry of the L-twist {000, 001, 100, 110} when c = 1.

All subgroups of this group are easily found by constructing the BDD for the Boolean function whose 24 variables are the potential symmetries. Indeed, all subsets of any set S that are closed under any given binary operator % on that set are the solutions to Λx,y∈S(¬x∨¬y∨(x*y)). In this case the resulting BDD (found in 2.5 Mμ) has 197 nodes, and it characterizes exactly 30 subgroups.

Two subgroups T and T ′ are said to be conjugate if T ′ = t–Tt for some t ∈ S. Such subgroups are considered to be equivalent, because they amount to viewing the objects from a different direction. The distinct conjugacy classes of subgroups according to this equivalence relation are called the “symmetry types,” and there are 11 of them: (i)

Class (ii) consists of the 12 symmetries whose permutations are even. The smallest polycube which admits these symmetries and no more — and hence it has just two base placements — contains 20 cubies, with 12 surrounding a central core of 8. Class (iv) has one symmetry for each permutation of the three coordinates. Classes (iii), (v), (vi), (vii), (ix), (x), (xi) correspond to the eight symmetry types of a square, with reflections implemented by “turning the square over” to the opposite side. In this interpretation biaxial symmetry becomes “tricentral,” because it corresponds to central symmetry about each coordinate axis. The former class called “180°” is now the same as “axial,” when viewed from either of the two other axes. [Many of these twelve examples have reflective symmetries too; but those don’t count. Under the full set of 48 hyperoctahedral symmetries, when reflections are allowed, there are 33 symmetry types(!), nicely presented by W. F. Lunnon in Graph Theory and Computing (Academic Press, 1972), 101–108. Lunnon also exhibited the ten symmetry types for polyhexes on pages 87–100.]

388. The directed path of four weak clues in (a) is equivalent to the five strong clues (1, 2, 3, 4, 5). Then there are “hidden singles” in columns 1 and 2, leading to a “naked single” in cell (4, 2), etc.; we cruise to victory without branching. Puzzle (b) has a naked single in (4, 2) — and we notice, by the way, that the middle cell needn’t be 5 even though it is greater than each of its four neighbors. Then (4, 4) is naked, and so on; again everything is forced. Puzzle (c) begins with hidden singles, which place the three missing 1s and then the 5 in row 0. After we fix cell (4, 2), the rest falls into place.

[Historical note: Futoshiki was invented by Yoshihiko Asao, who called it Dainarism (“Greater Than”); see Puzzle Communication Nikoli 92 (September 2000).]

389. In general, given a digraph in which each vertex v is supposed to be given an integer label l(v) with l(v) ≥ a(v), where the lower bounds a(v) have been specified, we can refine them as follows: For each vertex with d+(v) > 0, push v ⇒ S, where S is an initially empty stack. Then while S is nonempty, repeatedly do this: Pop S ⇒ v; for each w with v ––→ w and a(w) ≤ a(v), set a(w) ← a(v)+1, and push w ⇒ S if d+(w) > 0.

A similar algorithm will refine a given set of upper bounds b(v). For futoshiki, we apply these algorithms with a(v) = 1 and b(v) = n initially, except that a(v) = b(v) = l when a strong clue has specified v’s label. (Note: This method isn’t clever enough to prove that the middle element of puzzle (b) must be 3 or more. But it’s still very useful.)

390. In both cases we use primary items pij, rik, and cjk for 0 ≤ i, j < n and 1 ≤ k ≤ n, as we did for sudoku. There will be one option analogous to (30) for every (i, j) and for every k ∈ [aij ..bij ], where the bounds aij and bij are calculated as in exercise 389.

(a) Suppose there are w weak clues, where the tth weak clue is l(itjt) < l(i ′tj ′t). Introduce (n – 3)w secondary items gtd for 1 < d < n – 1 and 1 ≤ t ≤ w. Such an item informally means that l(itjt) > d and d ≥ l(i ′tj ′t); so we don’t want it to appear twice. We include gtd in each option for ij with d < k, and in each option for i ′j ′ with d ≥ k.

For example, the options for cells (0, 0) and (0, 1) in puzzle 388(b) are ‘p_00 r_02 c_02 g_12 g_13’, ‘p_00 r_03 c_03 g_13’, ‘p_00 r_04 c_04’, ‘p_00 r_05 c_05’; ‘p_01 r_01 c_11’, ‘p_01 r_02 c_12’, ‘p_01 r_03 c_13 g_12’, ‘p_01 r_04 c_14 g_12 g_13’. Another option is ‘p_22 r_23 c_23 g_23 g_33 g_43 g_53’.

(b) Introduce w primary items gt, and 3n^2 secondary items Pij, Rik, Cjk. The options for pij, rik, and cjk are ‘pij rik cjk Pij:k Rik:j Cjk:i’ for 0 ≤ i, j < n and aij ≤ k ≤ bij. The options for gt are ‘gt⁢  Pitjt:k⁢  Pit′jt′:k′⁢ Ritk:jt⁢ Rit′k′:jt′⁢  Cjtk:it⁢ Cjt′k′:it′’ for k < k ′, where k and k ′ are within the bounds for l(itjt) and l(it′jt′).

Experience shows that formulation (a) is a clear winner over formulation (b).

391. Given 5 ⋅ 5 ⋅ 5 options ‘pij rik cjk’ as in answer 390, Algorithm X needs just 230 megamems to generate 161280 = 5! ⋅ 4! ⋅ 56 solutions. [Euler enumerated them in his major paper on latin squares [Verhandelingen Genootschap Wetenschappen Vlissingen 9 (1782), 85–239, §148], though he was nearly blind at the time.] Every 5 × 5 latin square has 40 pairs of adjacent elements, leading to a string of 40 inequality signs; and we can sort those 161280 strings. Only 115262 distinct strings actually occur; and only 82148 of them occur just once. The other 79132 cannot be identified by weak clues only.

392. Here are the first examples found of each type, and the total number of cases:

(More detailed counting shows exactly (369404, 2976, 4216, 3584, ... , 80) cases with at least one long path and (0, 1, 2, 3, ... , 1344) solutions; (405636, 4000, 4400, 1888, ... , 72) cases with no long path and (0, 1, 2, 3, ... , 24128) solutions.) Example (i) below is one way to get the maximum number of solutions, using six particularly unhelpful clues.

The most interesting cases, of course, are those that make valid puzzles. They fall into equivalence classes under rotation and/or reflection and/or complementation; thus sixteen examples are typically equivalent to any given one. However, there are 46 equivalence classes with only eight members, self-dual under transposition, of which 26 have long paths (as in (ii), (iii), (iv) below) and 18 do not (as in (v), (vi), (vii)). Thus (173+26)+(241+18) = 458 essentially different futoshiki puzzles with six weak clues are valid; however, many of these are really the same, under row-and-column permutations that preserve all clues. The most difficult symmetric instance is probably (vii), because exercise 390 needs a 374-node search tree to solve it. (A clever solver will, however, deduce immediately that all diagonal elements of a symmetric puzzle must be 3!)

393. The 5^6 = 15625 ways to label six cells can be reduced to ω¯6=203, by limiting consideration to restricted growth strings (Section 7.2.1.5), multiplying the results for every such string by 5k when it has k different labels. (In fact, only 202 such strings are relevant, because the last one (123456) will be multiplied by 5^6 = 0 and never used.) Running through each subset of five cells, we find respectively (1877807500, 864000, 0, 0, 1296000, 10368000, ... , 144000) cases that have (0, 1, 2, 3, 4, 5, ... , 336) solutions.

Every case with a unique solution is obtained from the example shown by independently permuting the rows, columns, and labels. (Indeed, 864000 = 5!3/2.)

394. Let there be h strong clues and k = 5 – h weak clues. Four solutions are obtained only in (144, 2016, 2880) cases for h = (1, 2, 3). In every such case, two rows and two columns are completely free from clues; thus the four solutions arise from swapping those two rows and/or those two columns. As in answer 392, most of the cases belong to classes of 16 puzzles that are equivalent under rotation, transposition, and/or complementation. But when h = 3 there are 30 classes of size 8, having transposition symmetry (see (iii) and (iv) below); also 6 self-dual classes of size 8 (see (v)). Hence there are 9 + 126 + (36 + 162) = 333 inequivalent 4-solution 5-clue futoshikis altogether.

[This exercise was inspired by a talk that Dan Katz gave at the Joint Mathematics Meetings in January 2012. He observed, among other things, that valid puzzles exist with h + k = 6 for all values 0 ≤ h ≤ 6. Indeed, we can start with example (iv) in answer 392, and repeatedly insert a clue (5, 1, 5, 1, 4, 2) while removing an inequality.]

[The minimum number of strong clues needed to specify an n × n latin square is known to be ⌊n^2/4⌋ for n ≤ 8. See R. Bean, arXiv:math/0403005 [math.CO] (2004).]

395. Let L solve (vi) in answer 394. [See Appendix E if you’re stuck.] The only way to distinguish L from fifteen other latin squares that have the same string of 40 inequality signs is to give at least one clue 2 or 3 in a boundary row or column, at least one clue 4 or 5 in a boundary row or column, and at least one 4 or 5 in cells {(1, 1), (1, 3), (3, 1), (3, 3)}.

396. For example, here’s one that Algorithm P+X solves in 90 Mμ. (See Appendix E.)

397. (a) Assuming an m × n grid, let there be (m+1)(n+1) – 4 primary “endpoint” items ij for 0 ≤ i ≤ m, 0 ≤ j ≤ n, and [i =0] + [i = m]+ [j =0]+[j = n] ≤ 1; also “sheep” items sij when a sheep is in cell ij; also “start-stop” items + and –. Let there be mn secondary items xij for 0 ≤ i < m and 0 ≤ j < n, one for each cell. Three kinds of options are used: (i) There are 14(m–1)(n–1) “junction” options ‘ij x(i–1)(j–1):a x(i–1)j:b xij:c xi(j–1):d’, for 0 < i < m and 0 < j < n and 0 ≤ a, b, c, d ≤ 1 and (a = b or b = c or c = d). (ii) There are 2m +2n – 4 sets of four “boundary” options typified by ‘02 x_01:0 x_02:0’, ‘02 x_01:0 x_02:1 –’, ‘02 x_01:1 x_02:0 +’, ‘02 x_01:1 x_02:1’, for 0 ≤ i ≤ m, 0 ≤ j ≤ n, and [i =0]+[i = m]+[j =0]+[j = n] = 1; adjacent boundary cells, like x_01 and x_02 in this example, are listed in clockwise order. (For example, one of the options at the right boundary when n = 5 is ‘35 x_24:0 x_34:1 –’; one of the options at the left is ‘20 x_20:1 x_10:0 +’.) (iii) Each sheep has up to six “sheep” options, ‘sij xij:1 x(i–1)j:a xi(j+1):b x(i+1)j:c xi(j–1):d’, where a + b + c + d = 2; the x items are omitted if the corresponding cells lie outside of the grid, in which case their values are assumed to be 1. For example, the topmost sheep has only three options in the example puzzles, namely ‘s_03 x_03:1 x_04:b x_13:c x_02:d’, where b + c + d = 1.

This XCC problem for the rightmost example puzzle has five solutions:

To eliminate the spurious ones, we traverse the fence from ‘+’ to ‘–’, accepting a solution only if that path contains all of the color transitions between adjacent cells.

(b) There’s a unique solution if we put k sheep into a diagonal of length k; but that puzzle is trivial, not “interesting.” Random trials show that about one configuration in every 10,000 makes a suitable puzzle; the author found the first three examples below in that way. The fourth example was contrived by hand. All are solvable by hand:

[E. Olson invented this game; see J. Henle, Math. Intelligencer 40, 1 (2018), 69–70.]

398. The blanks in rows 0 and 4 of (c) can be filled with 3 and 5 in two ways.

[Tetsuya Miyamoto invented KenKen® in 2004, as an aid to education. The special case where all operations are multiplication, and all cages are rectangular, had been published by Ryuoh Yano in Puzzle Communication Nikoli 92 (September 2000).]

399. Set up an XCC problem with 3n^2 primary items pij, rik, cjk and 3n^2 secondary items Pij, Rik, Cjk, and with n^2 options ‘pij rik cjk Pij:k Rik:j Cjk:i’ for 0 ≤ i, j < n and 1 ≤ k ≤ n, as in answer 390(b). Also, if there are w cages, introduce primary items gt for 1 ≤ t ≤ w. Let Ct be the cells of the tth cage, and let there be an option

‘gt∪{{Pitjt:l(it,jt)Ritl(it,jt):jt,Cjtl(it,jt):it}|(it,jt)∈Ct}’

for every feasible way to assign labels l(it,jt) to the cells of Ct. For example, there are two labelings that satisfy the clue ‘15×’ in the third cage in puzzle 398(a), namely either l(0, 3) = 3 and l(0, 4) = 5 or l(0, 3) = 5 and l(0, 4) = 3; the two options for g_3 are therefore ‘g_3 P_03:3 R_03:3 C_33:0 P_04:5 R_05:4 C_45:0’ and ‘g_3 P_03:5 R_05:3 C_35:0 P_04:3 R_03:4 C_43:0’. The cage of that puzzle whose clue is ‘9×’ has just one option: ‘g_4 P_10:3 R_13:0 C_03:1 P_11:1 R_11:1 C_11:1 P_21:3 R_23:1 C_13:2’.

The option for a one-cell cage is trivial, and the options for two-cell cages are also easy. The options for larger cages are readily listed by a straightforward backtrack algorithm: We can represent unchosen labels in each row and column by bit vectors, just as unchosen values in the queens problem were represented in Algorithm 7.2.2B*. Simple upper and lower bounds on the final sum or product, given a partial labeling, yield satisfactory cutoffs in the analog to step B3* of that algorithm, based on the ρ and λ functions of Section 7.1.3. The ten-cell ‘34560×’ cage of puzzle 398(b) turns out to have 288 options, with 31 items each; the links will dance merrily around them all.

(Incidentally, this formulation doesn’t require the cells of a cage to be connected.)

400. The formulation in answer 399 makes it easy to omit the options for any cage. Thus Algorithm C almost instantaneously breezes through those 2048 problems, and finds that exactly 499 of them are uniquely solvable. The number of such puzzles with (5, 6, ... , 11) given clues is (14, 103, 184, 134, 52, 11, 1); for example, one can solve it when given only the clues ‘15×’, ‘6+’, ‘3–’, ‘5+’, ‘5’, in five cages! Exactly (14, 41, 6) of those 499 puzzles have (5, 6, 7) minimal clues; minimal-clue puzzles correspond to the prime implicants of the associated monotone Boolean function.

Similar remarks apply to puzzle 398(b), which can be solved uniquely without knowing either of the clues ‘34560×’ or ‘2’ — although the reader probably made heavy use of those clues when solving it. (On the other hand, its clue ‘9+’ cannot be omitted.)

401. There are 36 ways to cover a 4×4 board with dominoes, but nearly all of them are unsuitable. For example, can’t define the cages of a valid kenken problem, because the middle rows of any solution could be swapped to give another solution. And no two dominoes can cover a 2 × 2 region whose solution has the form abba. Therefore we’re left with only two cage patterns, and its transpose.

A given cage pattern can be filled with two clues of each type in 8!/2!4 = 2520 ways. Most of those ways are obviously impossible, because ÷ cannot be applied to the pairs {2, 3} or {3, 4}. It turns out that (1620, 847, 52, 1) of the cases give a kenken puzzle with respectively (0, 1, 2, 4) solutions. Notable examples are
and
where the first is the “most difficult,” in the sense that its search tree via the construction of exercise 399 has the most nodes (134). The second is the one with four solutions.

402. The solution follows answer 403. The author constructed this puzzle by first designing the cages, then generating a dozen or so random latin squares via exercise 86 until finding one that had a unique solution. Then the domino clues were permuted at random, ten times; the most difficult of those ten puzzles (77 meganodes) was selected.

The construction of answer 399 gives an XCC problem with 10914 options, 486 + 432 items, and 163288 total entries. There are respectively (720, 684, 744, 1310, 990, 360, 792, 708, 568, 1200, 606, 30) options for the pentominoes (O, P, ... , Z); preprocessing with Algorithm P reduces those counts to (600, 565, 96, 1122, 852, 248, 744, 656, 568, 1144, 606, 26). Overall, the reduced problem has 8927 options, 484 + 432 items, and 134530 total entries. The total time to find the solution and prove its uniqueness was 9 Gμ for Algorithm P and 293 Gμ for Algorithm C. (Without preprocessing, Algorithm C would have taken 6.4 Tμ, and its search tree would have had 2 giganodes. Could a human being solve this puzzle by hand?)

403. The author’s best attempt, shown below, manages to match 35 digits before deviating in the final cage. The construction of answer 399 fails spectacularly on this particular instance, because the monster cage for ‘79+’ has 3,978,616,320 options! We can, however, work around that problem by simply making row 7 unconstrained and subtracting 1+2+…+9 = 45 from the cage total. (A latin square is determined by any n – 1 of its rows.) Then Algorithm C solves the problem handily, with a cost of 2 Gμ (from 184422 options), and with a search tree of only 252 nodes. (See Appendix E.

Surprisingly, the non-π clue ‘3780×’ in the bottom row affects row 1 of the solution.)

404. Such puzzles can be defined on any N-vertex graph G, some of whose vertices are labeled with elements of {1, 2,...,N}; the problem is to extend such a labeling to a full Hamiltonian path, in all possible ways. We imagine an additional vertex ∞, which is adjacent to all the others. A Hamiltonian path in G is then equivalent to a Hamiltonian cycle in G ∪∞, with ∞ interposed between the first and last vertices of the path.

For 1 ≤ k ≤ N, let vk be the vertex labeled k, or vk = Λ if there’s no such vertex. Also let v_0 = vN+1 = ∞. We define an XCC problem with two kinds of primary items: (i) –v and +v for all unlabeled vertices v; (ii) sk for 0 ≤ k ≤ N, except when both vk = Λ and vk+1 = Λ. We also introduce secondary items pv for all unlabeled v, and qk for all unused labels k. (Thus the example has 35 primary items {–00, +00, –10, +10, –11, ... , +33, s_1, ... , s_7, s_9, ... , s_16}, and 20 secondary items {p_00, ... , p_33, q_2, q_4, ... , q_15, q_16}.) The options for sk are ‘sk –u pu:k qk:u +v pv:k+1 qk+1:v’ for all pairs of unlabeled vertices u ––– v such that u might be labeled k and v might be labeled k + 1. However, we omit –u pu:k qk:u if vk ≠ Λ, and we omit +v pv:k+1 qk+1:v if vk+1 ≠ Λ. For example, four of the options in the 4 × 4 toy problem are

‘s_3 +10 p_10:4 q_4:10’,     ‘s_6 –31 p_31:6 q_6:31 +30 p_30:7 q_7:30’,

‘s_4 –11 p_11:4 q_4:11’,     ‘s_6 –30 p_30:6 q_6:30 +31 p_31:7 q_7:31’;
the bottom two appear in the solution, but the top two do not. The secondary items are colored so that interdependent options will always link up properly.

Suppose l < k < r and vl ≠ Λ, vl+1 = … = vk = … = vr–1 = Λ, vr ≠ Λ. The statement “u might be labeled k” in the specification above means more precisely that there is a simple path of length k – l from vl to u and a simple path of length r – k from u to vr. (This condition is necessary for u to be labeled k, but not sufficient. It is, however, sufficient for our purposes.) A simple path of length 1 is equivalent to adjacency. A simple path of length > 1 can be decided using the algorithm in the following exercise; but if that algorithm is taking too long, we can proceed safely by assuming that a simple path does exist. The value of min(k – l, r – k) is usually small.

[Gyora Benedek invented Hidato® in 2005 and began to publish examples in 2008. Similar brainteasers sprang up later, based on other kinds of paths; but king moves Pm ⊠ Pn have a special appeal because they can cross each other.]

405. For l = 0, 1, ... , L, find the set Sl of all pairs (S, w) such that at least one simple path from v to w runs through the vertices of S ∪v, where S is an l-element set. Clearly S_0 = {(∅,v)}; and Sl+1 = {(S ∪ w, w) | w ––– u and w ∉ S for some (S, u) ∈ Sl}.

If at most 58 vertices w are reachable from v in ≤ l steps, we can represent each pair (S, w) in a single octabyte, with 6 bits for w and 58 bits for S. These octabytes can be stored in two stacks, alternately at the low and high ends of a sequential list.

406. The moves from 12 to 19 are forced, as are those on several other diagonals. So everything is quickly filled in, except for blanks between 42 and 51. Aha.

407. Using exercise 404, Algorithm C finds the 52 solutions quickly (1500 kilomems). Only one of them has ‘18’ in row 3, column 3; and that clue makes a puzzle with a nicely symmetric solution (see Appendix E). [We could also put ‘27’ in cell (2, 4); or ‘18’ in (4, 3); or ‘17’ in (4, 4). But that would destroy the smile.]

408.

409. Yes! (This puzzle is fiendishly difficult to solve by hand, although that has actually been done. Algorithm C finds the unique solution in 330 Mμ, with a search tree of 161612 nodes. (If you give up, the solution can be found in Appendix E.) A “pidato puzzle” like this is presumably possible only because 10 × 10 hidato solutions are quite abundant. Indeed, the actual number of 10 × 10 king paths is 721833220650131890343295654587745095696; it can be determined with ZDD technology, as explained in Section 7.1.4.)

410. Puzzles (a), (b), (d) have unique solutions; remarkably, all 12 of the clues in (b) are essential. But (c) has 40 solutions, including two whose loop doesn’t touch a corner.

Pattern (x), incidentally, has unique solutions for x = 0, 1, 2, but none for x = 3.

[Historical note: Slitherlink was invented by Nikoli editor Nobuhiko Kanamoto, who combined the puzzle ideas of Ayato Yada and Kazuyuki Yuzawa. See Puzzle Communication Nikoli 26 (June 1989).]

411. False; for instance, has two. [But such cases are somewhat mysterious. There are 93 of size 5 × 5, including three that give two loops despite 8-fold symmetry. A 6 × 6 example yields four loops; can you find them? (See Appendix E.) Are three loops possible? If m + 1 and n + 1 are relatively prime, N. Beluhov has proved that an m × n slitherlink diagram with all clues given cannot have more than one solution.]

412. With an m × n grid it’s convenient to use the (2m + 1)(2n + 1) pairs xy for 0 ≤ x ≤ 2m and 0 ≤ y ≤ 2n, with xy representing either (i) a vertex, if x and y are both even; (ii) a cell, if x and y are both odd; or (iii) an edge, if x + y is odd. The edge between two adjacent vertices is their midpoint. The four edges surrounding a cell are obtained by adding (±1, 0) and (0, ±1) to the coordinates of the cell.

To obtain the weak solutions for any slitherlink diagram on a planar graph, introduce one primary item for each vertex, one primary item for each face in which the number of edges is specified, and one secondary item for each edge. There are 1+(d2) options for each vertex υ of degree d, namely ‘υ e_1:x_1 … ed:xd’ where xj ∈ {0, 1} and x_1 + … + xd = 0 or 2. There are (dk) options for each face f of degree d that should have k edges in the path, namely ‘f e_1:x_1 … ed:xd’ with xj ∈ {0, 1} and x_1 + … + xd = k.

For example, the options for vertex 00 in the diagram of exercise 410(i) are ‘00 01:1 10:1’ and ‘00 01:0 10:0’. The options for cell 11 are ‘11 01:1 10:1 12:1 21:0’, ‘11 01:1 10:1 12:0 21:1’, ‘11 01:1 10:0 12:1 21:1’, ‘11 01:0 10:1 12:1 21:1’.

This construction yields (2, 2, 104, 2) weak solutions for puzzles 410(a) to 410(d). (In cases (a), (b), (d) we can delete or insert the 4-cycle that surrounds the middle cell.)

413. (Solution simplified by R. Molinari.) Let each record for an item include two new fields U and V. The U and V fields of a secondary item that represents edge u —— υ will point to the primary items u and υ. The U and V fields of a primary item that represents vertex υ are renamed MATE and INNER. MATE(υ) is zero until υ first becomes the endpoint of an edge, after which it points to the other endpoint of the path fragment containing that edge. INNER(υ) is nonzero when υ lies within a path fragment.

Introduce two new global variables: Global variable F is the current number of fragments. Global variable E is the edge that closed a loop, or zero if there’s no loop.

For example, suppose two edges currently have color 1, say υ_1 —— υ_2 and υ_3 —— υ_4. Then we’ve set MATE(υ_1) ← υ_2, MATE(υ_2) ← υ_1, MATE(υ_3) ← υ_4, MATE(υ_4) ← υ_3, and F ← 2. If now υ_2 —— υ_5 joins the fray, we set MATE(υ_5) ← υ_1, MATE(υ_1) ← υ_5, and INNER(υ_2) ← 1; but we leave MATE(υ_2) unchanged. Subsequent edges to υ_2 are rejected.

When the ‘purify’ routine (55) is called to give color 1 to a new edge i, it will refuse to do so when E is nonzero, because a loop has already been closed. Furthermore, when E = 0, it will know that edge i shouldn’t be chosen if U(i) and V(i) are mates and F ≠ 1, because that would close a loop disjoint from other fragments. On the other hand, it will close the loop if F = 1, also setting E ← i.

All of these operations are nicely and easily undone when we need to ‘unpurify’. For example, suppose edge i loses color 1 when u = U(i) and υ = V(i). If υ = MATE(u), we unclose the loop (and set E ← 0) if i = E; otherwise we zero the mates and set F ← F − 1. If MATE(u) ≠ MATE(υ), we set MATE(MATE(u)) ← u, MATE(MATE(υ)) ← υ, and INNER(u) ← INNER(υ) ← 0, F ← F + 1. The case MATE(u) = MATE(υ) is easy too.

Caution: Algorithm P must be modified so that it never discards redundant items, when it is used to preprocess a problem for this extension of Algorithm C.

414. After the forced moves have been made as shown, only two edges are undecided between the vertices of rows 1 and 2. A strongest possible algorithm will know that those two edges must either both be present or both absent. (In fact, a truly strongest possible algorithm will force both to be present as soon as any edge in or between rows 0 and 1 has been chosen.)

In general, consider the graph G consisting of the original vertices V and all the currently undecided edges. If X is any proper subset of V , connected or not, any loop will contain an even number of edges between X and V \ X. Thus any cutset of size two will force a relation between two undecided edges. An algorithm that dynamically maintains minimum cutsets of G (see Section 7.5.3) will therefore be helpful.

415. Instead of solving millions of puzzles, we can use the ZDD technology of Section 7.1.4 to list all the loops in P_6 □ P_6, of which there are 1222363. Say that the “signature” of a loop is the full sequence of 25 clues — the number of edges around each cell. It turns out that 93 pairs of loops have the same signature (see exercise 411); those 186 loops cannot be the solution to any 5 × 5 slitherlink puzzle. Let S be the set of 1222270 distinct signatures, and let S′ be the subset of 1222177 that give a valid 25-clue puzzle.

Suppose s′ ∈ S′ has t > 0 entries equal to digit d; and for s ∈ S let p(s, s′) be the binary “projection vector” x_1 … xt, where xk = 1 if and only if s has d in the kth cell where s′ has d. For example, if d = 1, the signatures s and s′ shown above have t = 10 and p(s, s′) = 1011101111. Form the set P (s′) = {p(s, s′) | s ≠ s′}. Then s′, with all clues restricted to digit d, is a valid puzzle if and only if 11 … 1 ∉ P (s′). Moreover, the valid puzzles contained in that one are precisely those whose projections aren’t contained in any element of P (s′). (If we regard P (s′) as a family of sets, such projections are the elements of ℘↗P(s′), in the notation of exercise 7.1.4–236.) We can find those vectors, and the minimal ones, with a reachability algorithm such as Algorithm 7.1.3R.

In this way we discover exactly (9310695, 833269, 242772, 35940, 25) valid puzzles for d = (0, 1, 2, 3, 4), of which exactly (27335, 227152, 11740, 17427, 25) have no redundant clues. The minimum number of clues, in such irredundant homogeneous puzzles, is respectively (7, 8, 11, 4, 1); and the maximum number is respectively (12, 14, 18, 10, 1). Many of the extreme cases make pleasant little puzzles:

(See Appendix E. This minimum-1s puzzle is one of two based on signature s′ above.)

416. Of course d = 4 is trivial. So is d = 0; but that case has an amusing sparse construction. The following puzzles generalize to all n with (n + d)mod 4 = 1:

(See the solutions in Appendix E.) N. Beluhov, who found these patterns for d = 2 and 3, has raised interesting problems of optimum density: Let β(d) = lim infn→ || S || /n^2 and β(d)¯ = lim supn →∞  || S || /n2, where S ranges over all valid n × n slitherlink puzzles that are d-homogeneous, and where || S || denotes the number of clues. Clearly || S || ≤ n^2/2 when d = 3, because no 2 × 2 subsquare can contain more than two 3s. Furthermore || S || ≥ n^2/4 − O(n) when d = 0. For we must eliminate at least n^2 + 2n of the 2n(n + 1) edges if all but one cycle is to be cut; each 0 eliminates at most four. If n > 5 we obtain a valid puzzle with only fourteen 1s, by placing a suitable 4×6 pattern in the upper left corner. Similarly, there’s a valid puzzle with only four 3s, if n > 3. Therefore these constructions prove that β(0)¯=β (1)¯ = β (2)¯ =1; β (3)¯ =1/2; β (1)¯ =β(3)¯=β(4)¯ =β(4)¯=0;β(0)¯ =1/4.

The intriguing case of β(2) remains unknown. Beluhov proved that it is at most 1116, using a construction for n = 4k that’s illustrated here for n = 12. Palmer Mebane has constructed this 2-homogeneous puzzle on an 8 × 8 board that has only 24 clues [see puzsq.jp/main/puzzle_play.php?pid=14178].

417. The pattern for d = 3 in answer 416 works also for d = 0, if we remove one clue from the top row. Fascinating diagrams arise when such patterns are attempted for d = 1; Beluhov’s largest example so far is the 30 × 30 puzzle obtained when removing the 1 in column 26 of row 0. (Such puzzles are extremely difficult for the algorithm of answer 413 to handle; but SAT solvers have no trouble with them.)

418. (a) 6 · 26^12 ≈ 5.7 × 10^17, from the central cell and 12 complementary pairs.

(b, c, d, e) As in answer 415, we define the projection p(s, s′) = x_1 …x_13, where xk = 1 if and only if s and s′ agree in the kth pair (or in the center, when k = 13). We obtain altogether 2,692,250,947 puzzles, of which 199,470,026 are minimal. The minimal ones include (1, 24, 0, 7, 42, 1648, 13428, 257105, … , 184, 8) that have respectively (1, 2, 3, 4, 5, 6, 7, 8, … , 19, 20) clues; here are some choice specimens:

419. To design this puzzle, the author began with the signature of the desired loop (see answer 415), then removed pairs of centrally opposite clues, more-or-less at random, until no redundant pairs remained. The construction of exercise 412 produced 2267 options on 404+573 items from the final clue set; and Algorithm P needed just 17 Mμ to remove 1246 of those options. Then the algorithm of exercise 413 discovered the solution, and proved it unique, with 5.5 Gμ of computation and a search tree of 15 meganodes. (It’s another big win for preprocessing: Otherwise that algorithm would have taken 37 Tμ, with a 78-giganode search tree!) Reference: D. E. Knuth, Computer Modern Type-faces (Addison–Wesley, 1986), 158–159.

420. (Solution by Palmer Mebane.) In any solution, each cell is either inside or outside the loop. Lemma: Every 2 has exactly two neighbors inside. (For if the 2 is outside, the neighbors opposite its two edges are inside; otherwise the neighbors opposite its two nonedges are inside.) Let S be the set of cells next to a 2. Color each 2 alternately red or blue; then each cell of S is a neighbor of exactly one red 2 and exactly one blue 2. In particular, that’s true for each inside cell of S. Thus, by the lemma, there are equally many red and blue cells. But that contradicts m mod 4 = n mod 4 = 1!

[This exercise is due to N. Beluhov, who observes that solutions aplenty exist when m is odd and n mod 4 = 3.]

421.

[Historical note: Masyu was invented by Ryuoh Yano, who developed a white-circles-only version, together with Mitsuhiro Ase, who contributed the black circles. See Puzzle Communication Nikoli 84 (April 1999); 90 (March 2000).]

422. Now we use the (2m − 1)(2n − 1) pairs xy for 0 ≤ x ≤ 2m − 2 and 0 ≤ y ≤ 2n − 2. Cell (i, j) corresponds to x = 2i and y = 2j (a “vertex”); clue (i, j) corresponds to x = 2i + 1 and y = 2j + 1. Edges are as before, and we use the same options to ensure that either 0 or 2 edges touch every vertex in a solution. The only essential change from answer 412 is the treatment of clues, since masyu clues are different from slitherlink clues.

A black masyu clue in (i, j) has four options, corresponding to north-west, northeast, south-west, and south-east legs; for example, the north-west option is

‘C(i, j) N(i, j):1 NN(i, j):1 W (i, j):1 WW (i, j):1’,
where C(i, j) = (2i+1)(2j+1), N(i, j) = C(i, j)−10, NN(i, j) = C(i, j) − 30, W (i, j) = C(i, j) − 01, WW (i, j) = C(i, j) − 03. Edges off the grid have “color” 0, so this option is omitted when i ≤ 1 or j ≤ 1.

A white masyu clue in (i, j) has six options, three for north-south orientation and three for east-west. The three for east-west are

‘C(i,j) E(i,j):1⁢ ⁢EE(i,j):0⁢ W(i,j):1⁢ WW(i,j):0’‘C(i,j) E(i,j):1⁢ ⁢EE(i.j):0⁢ ⁢W(i,j):1⁢ WW(i,j):1’‘C(i,j) E(i,j):1⁢E⁢E(i,j):1⁢W(i,j):1⁢ WW(i,j):0’

Again we omit an option that would set an off-board edge to 1. An off-board edge item that sets color 0 is silently dropped.

For example, the options for the black clue in exercise 421’s puzzle are ‘15 14:1 34:1 03:1 01:1’, ‘15 14:1 34:1 05:1 07:1’. The options for the white clue in the bottom row are ‘97 87:1 85:1 83:0’, ‘97 87:1 85:1 83:1’. That puzzle has 15 clue options altogether, and 119 vertex options ‘00 01:1 10:1’, ‘00 01:0 10:0’, ‘02 01:1 03:1 12:0’, … , ‘88 78:0 87:0’.

423. Obtain a representative of each class of equivalent variables, for example by adapting Algorithm 2.3.3E. This calculation may show that certain variables are constant. A contradiction might also arise — for example, if there’s a white clue in a corner; in such cases the masyu puzzle has no solution.

The vertex options of answer 422 can now be eliminated, at all vertices for which a clue was given. The clue options can also be consolidated, so that equivalent variables don’t appear together, and so that constants are suppressed. Every option that tries to set a variable both true and false is, of course, eliminated.

For example, variables 14, 50, 70, 85, and 87 in the puzzle of exercise 421 are forced to be true; variables 61 and 76 are forced to be false. We can eliminate variables 05, 16, 27, 36, 54, 65, and 74 because 05 = ∼03, 16 = 36 = ∼25, 27 = 25, 54 = 74 = ∼63, 65 = 63. The options for the black clue become ‘15 01:1 03:1 34:1’, ‘15 03:0 07:1 34:1’. The options for the white clue in the bottom row become ‘97 83:0’, ‘97 83:1’.

Caveat: These simplifications are very nice, but they mess up the single-loop-detection mechanism of answer 413 — because that answer uses several fields of item nodes as key elements of its data structure! To keep that algorithm happy, we must append a special option that covers all of the supposedly eliminated vertex items and constant-edge items; this option is ‘04 26 60 64 86 87:1 85:1 76:0 50:1 70:1 61:0 14:1’ in the example. We also need pairs of options such as ‘#25 16:1 36:1 27:0 25:0’ and ‘#25 16:0 36:0 27:1 25:1’, to keep all variables of an equivalence class in sync.

A tenfold speedup is achieved even on small puzzles like the 8 × 10 in exercise 426.

424. As in answer 415, we can begin with the 1222363 loops that are potential solutions. But this time the “signature” of a loop is the maximum set of clues that it supports. Such a signature turns out to have at most 24 clues; indeed, only puzzle (i) in Fig. A–5, along with its rotations or reflections, attains this maximum. (At the other extreme, 64 loops have an entirely empty signature, despite having lengths up to 28.)

Let S be the set of 905472 distinct signatures; and let S′ be the subset of 93859 that aren’t contained in (or equal to) the signature of any other loop. These are the signatures of loops that can solve a valid 6 × 6 puzzle. If s′ ∈ S′ has t clues, we define the projection vector p(s, s′) = x_1 …xt for s ∈ S by setting xj = 1 when s agrees with s′ in the jth cell where s′ has a clue. For example, when s′ is puzzle (i) and s is its transpose, the projection p(s, s′) is 000011000011101110110011.

Form the set P (s′) = {p(s, s′) | s ≠ s′}. We know that 11 … 1 ∉ P (s′), because s′ isn’t dominated by any other signature. Moreover, the valid puzzles having the loop of s′ as their solution are precisely those whose clues are not contained in any element of P (s′). We can find such puzzles, and the minimal ones, with a reachability computation like Algorithm 7.1.3R, whose running time is O(2t). For example, the loop of (i) turns out to be the solution to 8924555 puzzles(!). Four of them, such as (ii) and (iii), are minimal with only four clues; three of them, such as (iv), are minimal with eleven.

Most elements of S′ have far fewer than 24 clues. Hence it isn’t difficult to determine that there are exactly 1,166,086,477 valid 6 × 6 masyu puzzles altogether, of which 4,366,185 are minimal. (There are (80, 1212, 26188, 207570, … , 106) minimal puzzles with (3, 4, 5, 6, … , 12) clues. One of the 3s is puzzle (v); it also has the shortest loop. One of the 12s is puzzle (vi); it also has the longest loop — a Hamiltonian cycle. (A Hamiltonian cycle can actually be forced by only four clues; see puzzle (xvii).)

The valid puzzles include 5571407 that are pure white, 4820 that are pure black. The white clues can take on 22032015 different patterns; the black clues can assume only 39140. A surprisingly large number of 6 × 6 puzzles, 37472, can be “inverted,” remaining valid when white and black are swapped. If we restrict consideration to minimal puzzles, these figures become: 574815 pure white, 1914 pure black, 2522171 white patterns, 22494 black patterns, 712 invertible. The latter include many amusing and amazing pairs, such as (vii)–(viii), (ix)–(x), (xi)–(xii), as well as self-dual examples such as (xiii), (xiv), (xv), (xvi); there are 49 essentially distinct invertible puzzles of size 6 × 6. [Considerably larger invertible puzzles have been published in the anonymous blog uramasyu.blog80.fc2.com/, every few days since 2006.]

The author thinks puzzle (vi) may well be the hardest 6 × 6, although its search tree via exercise 423 has only 212 nodes. (That tree has 1001 nodes with exercise 422.)

425. A “balanced” n × n masyu solution of order k clearly requires 2 ≤ k ≤ ⌊n^2/4⌋. All such k turn out to be achievable, for n ≤ 6, except that the upper limit ⌊n^2/4⌋ is not. Solutions for k = 2 exist for all n ≥ 3; solutions for k = 3 exist for all n ≥ 4; solutions for k = 4, due to B. S. Ho, exist for all n ≥ 5; solutions for k = 5 and k = 6, due to G. J. H. Goh, exist for all n ≥ 6. (See (xviii)–(xxiv) in Fig. A–5.) Goh has also discovered analogous constructions for k = 7, 8, 9, 10.

426. The clue in the corner must obviously be ‘ ’. That leaves us with 2^28 other possibilities to consider, many of which can be rejected immediately because certain local patterns are impossible. (For example, there cannot be three consecutive ‘ ’s.) Consider the Boolean function of x_0x_1 …x_27 that’s true if and only if the diagram has at least one solution, with ‘ ’ when xj = 1 and ‘ ○ ’ when xj = 0. One can easily verify that there’s no solution when x_0x_1 or x_1x_3 or x0x¯1x4 or … or x¯3x¯4x¯5 or x_6x_7 or x¯7x¯8x¯10x¯11, etc.; also when we replace xj by xj+12. We can also rule out extreme cases such as x¯1x¯26.

Fig. A–5. A gallery of interesting 6 × 6 masyu puzzles.

After compiling several dozen such “bad” configurations, the author applied BDD technology: Less than a megamem sufficed to generate a BDD of size 715, which showed that exactly 10239 vectors x_0x_1 … x_27 were not yet ruled out. The masyu solver of exercise 423 tossed off those cases with search trees of 3 nodes per problem, on average; and it turned out that exactly (10232, 1, 1, 1, 4) vectors had (0, 1, 2, 3, 4) solutions. The unique winning puzzle is shown here (and solved in Appendix E).

427. Here’s an example with 8 · 15 white clues (solved in Appendix E):

It turns out to be problematic for the method of exercise 423, which severely loses focus and takes forever to prove that there’s only one solution. One can, however, exploit symmetry by modifying Algorithm C as follows: Whenever a color setting is made on the rightmost branch of the search tree, all settings that are equivalent to it by symmetry can be forced. Then uniqueness is proved in about 36 Mμ, provided that the primary items are suitably ordered. [This exercise was inspired by Nikoli’s Giant Logic Puzzles for Geniuses (Puzzlewright Press, 2016), #53.]

428. (Solution by N. Beluhov.) 3n − 12 black clues suffice when n mod 4 = 0; 5n − 21 white clues suffice when n mod 4 = 1. (Are these constants 3 and 5 the best possible?)

429. (a) Incidentally, each of these puzzles is minimal (all clues important):

(b) In fact, two permutations of the colors are possible in each case:

430. (a) The lower right corner must contain 5. See Appendix E for the other cells.

(b) Set cnk ← 0 for all n and k. Now do this for 3 ≤ x < 512: Set k ← n ← 0; for 0 ≤ t < 9 set k ← k +1, n ← n + t +1 if x &(1 ≪ t) ≠ 0; finally if k > 1, set Cnkcnk ← x and cnk ← cnk + 1. The n-in-k combinations are now Cnkj for 0 ≤ j < cnk.

The maximum cnk, 12, is obtained for (n, k) = (20, 4) or (25, 5). Notice that cnk = c(45−n)(9−k) when 1 < k < 8. Cases with cnk = 1 are called “restricted” or “magic blocks”; they’re extremely helpful when present (but our example doesn’t have any).

(c) The middle must be 7 9 8 (an odd digit < 9, 9, then an even digit).

(d) The tables from (b) convert kakuro to generalized kakuro. Introduce a primary item ij for each cell to be filled. Let there be H horizontal blocks, and assume that horizontal block h has ch combinations Xhp of length kh, for 1 ≤ h ≤ H and 1 ≤ p ≤ ch. Introduce chkh primary items Hhpx, for x ∈ Xhp, to represent the elements of block h’s pth combination. (For example, the primary items for the first horizontal block of our example are H_111, H_114, H_122, H_123 because the two combinations are {1, 4} and {2, 3}.) Similarly, introduce primary items Vυqy for the elements of the qth combination Yυq of vertical block υ, for 1 ≤ υ ≤ V and 1 ≤ q ≤ dυ.

Also introduce secondary items Hh and Vυ for 1 ≤ h ≤ H and 1 ≤ υ ≤ V , one for each block. The “color” of such an item represents the choice of combination to be used.

The options for cell ij are ‘ij Hhpx Hh:p Vυqx Vυ:q’, where h and υ indicate the horizontal and vertical blocks through ij, for 1 ≤ p ≤ ch and 1 ≤ q ≤ dυ and x ∈ Xhp ∩ Yυq. (Thus, the options for the upper left blank cell in our example are ‘11 H_111 H_1:1 V_111 V_1:1’, ‘11 H_114 H_1:1 V_124 V_1:2’, ‘11 H_122 H_1:2 V_122 V_1:2’. Set intersections are easily computed from the bitmaps Xhp and Vυq.)

Additional options are also necessary to “absorb” the combinations not used. These are ‘ ∪{Hhpx | x ∈ Xhp} Hh:p′’ for 1 ≤ p, p′ ≤ ch and p ≠ p′; ‘∪{Vυqy | y ∈ Yυq} Vυ:q′’ for 1 ≤ q, q′ ≤ dυ and q ≠ q′. (Thus the options for h = 1 in our example are ‘H_111 H_114 H_1:2’, ‘H_122 H_123 H_1:1’.) This instructive construction deserves careful study.

431. There are 18 solutions, because of two ways to complete the middle left portion and (independently) nine ways to complete the lower left corner. (The digits that are uniquely determined by his conditions are shown below.) We can freeze most of those digits, and extract two much smaller problems, then insert a few wildcards as in exercise 433 until obtaining uniqueness. One suitable patch, shown below, changes seven clues and has the solution found in Appendix E. (In this problem, preprocessing greatly improves the focus, reducing the search tree size from 115 million to just 343!)

[Funk had copyrighted a Cross Sums Puzzle already in September 1935; see Canadian Patent Office Record and Register of Copyrights and Trade Marks 63 (1935), 2253.]

432. (a) We save a lot of time by considering only “restricted growth strings” as solutions (see Section 7.2.1.5). That is, we can assume that the top row is ‘12’; then the second row is either ‘213’ or ‘234’ or ‘312’ or ‘314’ or ‘34x’ for 1 ≤ x ≤ 5; etc. Altogether there are (5, 28, 33, 11, 1) such strings with maximum element (3, 4, 5, 6, 7). Thus we know that the blanks can be filled in 5 ·93+28 ·94+33 ·95+11·96+97=1432872 ways. And we can quickly compute the 1432872 sequences of block sums from those restricted growth strings, using a table of 9! permutations built by Algorithm 7.2.1.2L. Exactly 78690 of those sequences, about 5.5%, occur uniquely and define a kakuro puzzle.

Every kakuro puzzle has a dual, obtained by replacing all clue-sums s for blocks of length k by 10k − s; the dual is solved by changing each digit d to 10 − d. Thus, if a puzzle of type (a) is defined by horizontal and vertical sums s_1s_2s_3/t_1t_2t_3, its dual is defined by (20−s_1)(30−s_2)(20−s_3)/(20−t_1)(30−t_2)(20−t_3). Diagonal symmetry also makes s_1s_2s_3/t_1t_2t_3 equivalent to s_3s_2s_1/t_3t_2t_1 and t_1t_2t_3/s_1s_2s_3; so we get up to eight equivalent puzzles from each sequence. There are 9932 essentially distinct puzzles, only one of which has four symmetries, namely 6 15 14/14 15 6; 190 have one symmetry, and the remaining 9741 are asymmetric. (The asymmetric ones are, of course, more difficult to solve, because a symmetric puzzle will have a symmetric solution.) The example 5196/6 10 14 in exercise 430 is asymmetrical; but it’s relatively easy because it has a forced move in the lower right corner. The easiest puzzles, with four forced moves, are 41512/12 15 4 and 4 15 16/12 15 8, both symmetric. Altogether 4011 of the asymmetric puzzles have no forced moves. And of those, 570 have no “magic blocks.” And of those, puzzle 6 19 6/8 11 10 is the hardest, in the sense that it maximizes the number of nodes (79) in Algorithm C’s search tree, using the construction of answer 430(d).

(b) Similarly, this shape has 2·93+42 ·94+186 ·95+234·96+105 ·97+18 ·98+99=43038576 sequences of block sums, of which 6840 ≈ 0.016% are unique. Those 6840 yield 49 equivalence classes under the symmetries s_1s_2s_3/t_1t_2t_3 ↦ s_2s_1s_3/t_1t_2t_3, s_3s_2s_1/t_1t_2t_3, s_1s_2s_3/t_2t_1t_3, s_1s_2s_3/t_3t_2t_1, t_1t_2t_3/s_1s_2s_3, (30−s_1)(30−s_2)(30−s_3)/ (30−t_1)(30−t_2)(30−t_3). All but 3 of those 49 puzzles are asymmetric; 7 11 20/71120 and 71920/7 19 20 are self-transpose, and 7 15 23/10 15 20 is self-dual. They aren’t great, because they all have at least one forced move from 7 opposite 20 or from its dual.

[It’s extremely difficult to find a kakuro puzzle whose spaces make a 4 × 4 grid. But Johan de Ruiter discovered in 2010 that there are five essentially different ways. For example, 11 15 23 29/12 15 23 28 has a 488-node search tree, so it’s a nice little challenge.]

433. A slight extension to the construction of answer 430(d) allows “wildcard” blocks, with unspecified length and with the universal combination {1,..., 9} as their X or Y . The items Hh_1x or Vυ_1y for such wildcards are secondary, not primary. Algorithm C now pumps out 89638 solutions (in 150 Mμ); and 12071 of the corresponding sum sequences s_1 …s_7/t_1 …t_7 occur once only and yield valid puzzles. (The easiest ones, 16418(d+14) 16 16 16/934246 d 12 15 for 7 ≤ d ≤ 9, have a search tree of only 47 nodes. A median puzzle such as 16 4 20 18 16 16 15/9 22 24 6 17 12 15 needs 247 nodes. And the hardest, 16 4 23 19 16 16 13/9 25 24 6 17 11 15, needs 1994.)

[The author tried 10000 experiments in which all 21 cells of this diagram were simply filled at random, and their block sums recorded. Those 10000 problems had ≈ 75 solutions, on average, with standard deviation ≈ 1200. Only five of them led to valid puzzles; the most difficult one, 15 3 21 16 27 8 10/9 22 28 11 21 5 4, needed 1168 nodes.]

434. In 700 Mμ, a BDD with 64 variables and 124487 nodes characterizes 93,158,227,648 solutions. N. Beluhov proved in 2018 that there are at most 38 blocks, achieved for example as shown, by listing all cases with 38 or more. He also observed that the maximum for n × n kakuro is n^2/3 − O(n), using a similar construction with (i, j) black ⇔ (i + j) mod 3 = 0 except near the boundary.

435. The search tree for this one has 566 nodes. (See Appendix E.)

436. (a) Any solution with a black seed works also with that cell white.

(b) A solution with a non-articulation point would work also with that cell black.

437. Introduce a primary item ∗ to make the seeds white; also primary items Ric and Cjc for each character c that occurs more than once in row i or column j. Introduce secondary items ij for 0 ≤ i < m and 0 ≤ j < n, representing cell (i, j). For example, the first option for puzzle 436(α) is ‘∗ 01:0 02:0 10:0 13:0 14:0 21:0 31:0 32:0’.

Suppose row i contains character c in columns j_1, … , jt, where t > 1. Then Ric normally has t + 1 options ‘Ric ij_1:e_1 … ijt:et u_1:0 … us:0’ for e_1 + … + et ≥ t − 1, where {u_1,...,us} are the non-seed neighbors of the cells being colored 1. However, this option is suppressed if it would assign two colors to the same item. For example, if i = 1, t = 3, and j_1j_2j_3 = 123, there is only one option ‘R1c 11:1 12:0 13:1 01:0 03:0 10:0 14:0 21:0 23:0’ (but with entries deleted that color a seed with 0), because the other three options are contradictory.

Of course the options for Cjc are similar. For example, the options for C3L in puzzle 436(α) are ‘C3L 23:0 33:1 34:0’ and ‘C3L 33:0 23:1 22:0 24:0’.

[Notice, incidentally, that this XCC problem is a special case of 2SAT. Therefore it can be solved in linear time. Furthermore, by Theorem 7.1.1S, the median of any three solutions is also a solution — a curious fact!]

438. The basic idea is to abandon partial solutions that cut off any white cells from the first seed. Connectedness can be assured by maintaining a triply linked spanning tree, rooted at that seed, with the help of new fields in each item record. Changes to the spanning tree need not be undone when unblackening a cell while backtracking; any spanning tree on the currently nonblack cells is satisfactory.

[This method can be patched to handle the rare instances that have no seeds. To ensure uniqueness, as in exercise 436(b), each solution should also be tested for articulation points. Hopcroft and Tarjan’s algorithm for bicomponents does that efficiently. See Section 7.4.1.2; also The Stanford GraphBase, pages 90–99.]

439. (a) Property (ii) states that U is a vertex cover (or equivalently that V \ U is independent). Thus (i) and (ii) together state that U is a connected vertex cover. Adding property (iii) gives us a minimal connected vertex cover. [Minimal connected vertex covers were introduced by M. R. Garey and D. S. Johnson in SIAM J. Applied Math. 32 (1977), 826–834, who proved that it is NP-complete to decide if a planar graph with maximum degree 4 has a connected vertex cover of a given size.]

(b) This is the thrust of exercise 436(b). [N. Beluhov has proved constructively that every m × n hitori cover for m, n > 1 solves at least one valid puzzle, using an alphabet of at most max(m, n) letters.]

440. False (if neither A is alone in its column). Consider or .

441. When n = 1 any single letter a is trivially a valid puzzle. When n > 1 the possibilities are (i) aαa for every string α of n − 2 distinct letters containing an a (thus (n − 2)dn − 2 puzzles); (ii) aαb for a ≠ b and every string α of n − 2 distinct letters containing a and b (thus (n − 2)2 dn − 2 puzzles); altogether (n − 2)2dn −2 valid puzzles.

442. A “frontier-based” algorithm analogous to those of answers 7.1.4–55 and 7.1.4–225 will produce an unreduced ZDD for the family f of all complements V \ U of connected vertex covers, from which a variant of Algorithm 7.1.4R will give a ZDD. Then the NONSUB subroutine of answer 7.1.4–237 will produce a ZDD for f↑, the complements of hitori covers (the black cells of potential solutions). In the most complicated case, m = n = 9, an unreduced ZDD of size 203402 is reduced quickly to 55038 nodes; then 550 G μ of computation produces a ZDD of size 1145647 for the family of maximal black cells.

Those ZDDs make it easy to count and generate hitori covers; we obtain the totals

(12111111124612203664112200161130751734341054255811230110382127042981456049204120753821804788836627166217755680136173127078884641628768517511541065681416443442983662728768523934221936641115755721811121054145601662171751154193664112089750422255742067120025584920475568010656814157557218225574206732411910059)

Further statistics about these fascinating patterns are also of interest:

The left-hand matrix shows how many black cells can occur in hitori covers. The right-hand matrix shows how many hitori covers have both horizontal and vertical symmetry; when m ≠ n, such covers are counted just once in the previous totals, while the unsymmetrical covers are counted twice or four times. When m = n, such covers are counted either once (if there’s 8-fold symmetry) or twice (otherwise); there are respectively (1, 0, 1, 0, 2, 0, 2, 0, 11) n × n hitori covers with 8-fold symmetry. Further types of 4-fold symmetry are possible when m = n: There’s 90° rotational symmetry (but not 8-fold) in (0, 0, 0, 1, 1, 3, 11, 30, 106) pairs of cases; there’s symmetry about both diagonals (but not 8-fold) in (0, 0, 0, 0, 0, 1, 4, 9, 49) pairs of cases. Figure A–6 shows some of the winners in this beauty contest for symmetrical hitori covers.

Fig. A–6. A gallery of interesting hitori covers.

Fourfold horizontal and vertical symmetry is impossible when m and n are both even, because it forces at least 12 white cells near the center. The number of 2 × n hitori covers can readily be shown to satisfy the recurrence Xn = 2Xn−2 + 2Xn−3, growing as Θ(rn) where r ≈ 1.76929.

443. (Solution by N. Beluhov.) Let there be s black cells, of which a lie in the interior, b on the boundary but not in a corner, and c in a corner. One can show that b + 2c ≤ m + n + 2 − [m even] − [n even] − [mn odd]. Therefore the number of edges in Pm □ Pn | U is m(n − 1) + (m − 1)n − 4a − 3b − 2c = 2mn − m − n − 4s + b + 2c ≤ 2mn − 4s + 1. But Pm □ Pn | U is connected, so it has at least mn − s − 1 edges.

[Beluhov has also proved that the number of black cells is always at least mn/5 − O(m + n). One can obtain a small hitori cover by blackening (i, j) when i + 2j is a multiple of 5, and possibly a few more cells; this cover has at most mn/5 + 2 black cells.]

444. No. By exercise 443, the solution has at most ⌊(n^2/ 3 + 2)/n⌋ black cells in some row. This is at most n/3, when n > 5; hence 2n/3 elements of that row are white. Conversely, the puzzle illustrated here for n = 9 can be generalized to 3k ×3k for all k > 1. (It’s a simplification of a construction by N. Beluhov. Notice that every nonzero element is a seed!)

445. Array (α) below is a seedless puzzle that corresponds to (ii), if you change its lowercase letters to uppercase. (The lowercase letters are convenient for our purposes in understanding seedlessness, because they indicate the cells that we’ll want to darken.) When every black cell has a different letter to be hidden, a seedless puzzle must fill each white cell (i, j) with a hidden letter from either row i or column j.

Given a hitori cover, its “RC problem” is to put either R or C into each white cell so that the number of Rs in each row is at most the number of black cells in that row, and the number of Cs is similar but for columns. Array (β) shows the RC solution that corresponds to (α); this is one of four ways to solve the RC problem for (ii).

Suppose a hitori cover has s black cells. Every solution to its RC problem has at most s white cells marked R and at most s marked C; so we must have s ≥ n^2/3 in an n × n cover. Consequently s must be 12 when n = 6, by exercise 443. In particular, pattern (i) can’t lead to a seedless puzzle. Also, equality must hold when we said “at most.”

It’s easy to formulate the RC problem as an MCC problem, by introducing a primary item ij for each white cell (i, j), also primary items Ri and Cj for each nonwhite row i and column j. In the problem for pattern (ii) we have, for example, two options ‘23 R_2’ and ‘23 C_3’ for item 23. The multiplicity of C_3 is 2. (This is actually a bipartite matching problem; we use Algorithm M only because of the multiplicities.)

Array (γ) shows a seedless puzzle different from (α) that comes from the same RC solution (β). Indeed, (β) yields 3!1!2!2!1!3! · 3!1!2!2!1!3! = 20736 different seedless puzzles, because the letters chosen in each row and column can be permuted arbitrarily.

All such permutations yield valid puzzles. Proof: Each of the 12 letters occurs thrice. To solve the puzzle we must blacken each letter at least once, preserving white connectedness. One successful solution is to kill two birds with each stone; any other way would blacken 13 or more. But no 6 × 6 hitori cover has more than 12 black cells.

Pattern (iii) has eight RC solutions, and 20736 seedless puzzles for each of them.

Pattern (iv) has no RC solutions. But pattern (v) has the unique solution (δ), and one of its 3!0!3!2!1!3! · 2!2!1!3!1!3! = 62208 seedless puzzles is (∈).

[N. Beluhov has proved that valid n × n seedless puzzles exist ⇔ n mod 6 = 0.]

446. There are only 1804 hitori covers, according to answer 442; but the exact probability appears to be difficult to compute. Experiments with millions of random numbers show convincingly, however, that the probability is ≈ .0133. It drops to ≈ .0105 with radix 8, and even further to ≈ .0060 with radix 16; the “sweet spot” appears to be radix 10(!). [Also, the probability for decimal 4 × 4 is ≈ .0344; for 6 × 6 only ≈ .0020.]

447. Yes, when 2 ≤ m ≤ 4 and n = 6! (Johan discovered the 4 × 6, and the 5 × 5 for e, in 2017. The cases 2 × 6, 3 × 2, and 4 × 5 also work for e. By exercise 443, we can assume that m, n ≤ 15.)

448. There are just two answers. (Also a nice 6 × 6 with only one not-so-common word.)

449. A few more nuggets: Johan noticed (i) in the (appropriately named) 1990 movie Home Alone; and he found (ii) in the King James Bible, Luke 9 : 56. George Sicherman hit on Falstaff’s famous repartee (iii) in 1 Henry IV, Act V, Scene 4, Line 119. The author found (iv) within the graffiti on page 278 of CMath; also (v), an inspiring remark by Francis Sullivan, on page 2 of Computing in Science and Engineering 2,1 (January/February 2000). Example (vi) appears in the front matter to Volume 1. And example (vii), also 11 × 3, shows that a nice hitori can involve lowercase letters, spaces, and punctuation; it’s a quote from Samuel Rogers’s poem Human Life (1819).

The current record for largest literary hitori nugget, 12 × 5 = 60, was found by Gary McDonald in September 2019: “Ruth intimated that, as far as she could judge, he was a very eligible swain.” [Charles Dickens, Martin Chuzzlewit.]

450. The solutions are characterized by 25 items {tot, tibi, … , caelo, 1a, 1b, 1c, … , 5a, 5b, 5c, 6a, 6b} and 80 options ‘tot 1a’, ‘tot 1b 1c’, … , ‘tot 4b 4c’, ‘tot 5a’, ‘tot 6a’, ‘tot 6b’; ‘tibi 1b 1c’, ‘tibi 1c 2a’, … , ‘tibi 5c 6a’; … , ‘sidera 1a 1b 1c’, … , ‘sidera 5a 5b 5c’; ‘caelo 1a 1b 1c’, ‘caelo 1b 1c 2a’, … ,‘caelo 4b 4c 5a’, ‘caelo 6a 6b’.

Section 7.2.2.2

1. (a) ∅ (no clauses). (b) {∈} (one clause, which is empty).

2. Letting 1 ↔ lazy, 2 ↔ happy, 3 ↔ unhealthy, 4 ↔ dancer, we’re given the respective clauses {314,1¯42,3 4¯2,2¯⁢43¯,1¯⁢32¯,23¯⁢1,1¯⁢43¯},, matching R′ in (7). So all known Pincusians dance happily, and none are lazy. But we know nothing about their health. [And we might wonder why travelers have bothered to describe so many empty sets.]

3. f(j − 1, n) + f(k − 1, n), where f(p,n)=Σd=1q(n−pd)=p(q2)+q(n⁢  modp)≈n2/(2p), if we set q = ⌊n/p⌋.

4. Those constraints are unsatisfiable if and only if we remove a subset of either {357,456,3¯⁢5¯⁢7¯,4¯⁢5¯⁢6¯},{246,468,2¯⁢4¯⁢6¯,4¯⁢6¯⁢8¯},{246,357,468,4¯⁢5¯⁢6¯⁢} or {456,2¯⁢4¯⁢6¯,3¯⁢5¯⁢7¯,4¯⁢6¯⁢8¯}.

5. No polynomial upper bound for W (3,k) is currently known. Clearly W (3,k) is less than W¯⁢(3,k), the minimum n that guarantees either three equally spaced 0s or k consecutive 1s. An analysis by R. L. Graham in Integers 6 (2006), A29:1–A29:5, beefed up by a subsequent theorem of T. F. Bloom in J. London Math. Society (2) 93 (2016), 643–663, shows that W¯⁢(3,k)=exp O(k(logk)4)).

6. Let each xi be 0 with probability p = (2ln k)/k, and let n be at most k^2/(ln k)3. There are two kinds of “bad events”: Ai, a set of three equally spaced 0s, occurs with probability P = p^3; and A′j, a set of k equally spaced 1s, occurs with probability P′ = (1 − p)k ≤ exp(−kp) = 1/k^2. In the lopsidependency graph, which is bipartite, each Ai is adjacent to at most D = 3k^3/((k − 1)(ln k)3) nodes A′j; each A′j is adjacent to at most d=32k3/(lnk)3 nodes Ai. By Theorem L, we want to show that, for all sufficiently large values of k, P ≤ y(1 − x)D and P′ ≤ x(1 − y)d, for some x and y.

Choose x and y so that (1 − x)D = 1/2 and y = 2P . Then x = Θ((log k)3/k^2) and y = Θ((log k)3/k^3); hence (1 − y)d = exp(−yd + O(y^2d)) = O(1). [See T. Brown, B. M. Landman, and A. Robertson, J. Combinatorial Theory A115 (2008), 1304–1309.]

7. Yes, for all n, when x_1x_2x_3 … = 001001001 ….

8. For example, let xi,a signify that xi = a, for 1 ≤ i ≤ n and 0 ≤ a < b. The relevant clauses are then xi,0 ∨ … ∨ xi,b−1 for 1 ≤ i ≤ n; and x¯i,a∨x¯i+d,a⁢⁢∨…∨x¯i+(ka−1)d,a for 1 ≤ i ≤ n − (ka − 1)d and d ≥ 1. Optionally include the clauses x¯i,a∨x¯i,a′ for 0 ≤ a < a′ < b. (Whenever the relevant clauses are satisfiable, we can also satisfy the optional ones by falsifying some variables if necessary.)

[V. Chvátal found W (3, 3, 3) = 27. Kouril’s paper shows that W (2, 4, 8) = 157, W (2, 3, 14) = 202, W (2, 5, 6) = 246, W (4, 4, 4) = 293, and lists many smaller values.]

9. W (2, 2,k) = 3k − (2, 0, 2, 2, 1, 0) when k mod 6 = (0, 1, 2, 3, 4, 5). The sequence 2k−1 02k−1 12k−1 is maximal when k ┴ 6; also 2k−102k−112k−3 when k mod6 = 3; also 2k−102k−212k−1 when k mod 6 = 4; otherwise 2k−102k−212k−2. [See B. Landman, A. Robertson, and C. Culver, Integers 5 (2005), A10:1–A10:11, where many other values of W (2,..., 2,k) are also established.]

10. If the original variables are {1,...,n}, let the new ones be {1,...,n}∪{1′,...,n′}. The new problem has positive clauses {11′,...,nn′}. Its negative clauses are, for example, 2¯′6¯⁢7¯⁢9¯′ if 26¯⁢7¯⁢9 was an original clause. The original problem is equivalent because it can be obtained from the new one by resolving away the primed variables.

[One can in fact construct an equivalent monotonic problem of size O(m + n) in which (x_1 ∨ … ∨ xk) is a positive clause if and only if (x¯1∨…∨x¯k) is a negative clause. Such a problem, “not-all-equal SAT,” is equivalent to 2-colorability of hyper-graphs. See L. Lovász, Congressus Numerantium 8 (1973), 3–12; H. Kleine Büning and T. Lettmann, Propositional Logic (Cambridge Univ. Press, 1999), §3.2, Problems 4–8.]

11. For each variable i, the only way to match vertices of the forms ij′ and ij″ is to choose all of its true triples or all of its false triples.

Furthermore, the only way to match j′1 is to choose one of the satisfiability triples for clause j. Suppose l¯kj belongs to the chosen triple; then we must also have chosen the true triples for literal lk. Thus a perfect matching implies satisfiable clauses.

Conversely, if all clauses are satisfied, with lk true in clause j, there always are exactly two ways to match l¯kj with j′1 while matching ωj, xj, yj, zj, and the other two l¯j vertices with j′2, …, j′7. (It’s a beautiful construction! Notice that no vertex appears in more than three triples.)

12. Equation (13) says S_1(y_1,...,yp) = S≥1(y_1,...,yp) ∧ S≤1(y_1,...,yp). If p ≤ 4, use ∧1≤j<k≤p (y¯j∨y¯k) for S≤1(y_1,...,yp); otherwise S≤1(y_1,...,yp) can be encoded recursively via the clauses S≤1(y1,y2,y3,t)∧S≤1(t¯,y4,…,yp), where t is a new variable. [This method saves half of the auxiliary variables in the answer to exercise 7.1.1–55(b).]

Note: Langford’s problem involves primary items only; in an exact cover problem with nonprimary items, such items only need the constraint S≤1(y_1,...,yp).

13. (a) S_1(x_1, x_2, x_3, x_4, x_5, x_6) ∧ S_1(x_7, x_8, x_9, x_10, x_11) ∧ S_1(x_12, x_13) ∧ S_1(x_14, x_15, x_16) ∧ S_1(x_1, x_7, x_12, x_14) ∧ S_1(x_2, x_8, x_13, x_15) ∧ S_1(x_1, x_3, x_9, x_16) ∧ S_1(x_2, x_4, x_7, x_10) ∧ S_1(x_3, x_5, x_8, x_11, x_12) ∧ S_1(x_4, x_6, x_9, x_13, x_14) ∧ S_1(x_5, x_10, x_15) ∧ S_1(x_6, x_11, x_16).

(b) Duplicate clauses occur when options intersect more than once. We avoid them if we simply generate clauses x¯i∨x¯j for every pair (i, j) of intersecting options.

(c) When langford (4) is generated in this way, it has 85 distinct clauses in 16 variables, namely (x1∨x2∨x3∨x4∨x5∨x6)∧(x7∨x8∨x9∨x10∨x11) ∧ … ∧ (x6∨x11∨x16)∧(x¯1∨x¯2)∧(x¯1∨x¯3)∧…∧(x¯15∨x¯16).

But langford′ (4) cannot use the trick of (b). It has 85 (nondistinct) clauses in 20 variables, beginning with 123456,1¯⁢2¯,1¯⁢3¯,1¯⁢1¯′,2¯⁢3¯,2¯⁢1¯′,3¯⁢1¯′,1′4¯,1′5¯,1′6¯,4¯5¯,4¯⁢6¯,5¯⁢6¯,…, if we denote the auxiliary variables by 1′, 2′, …. Two of those clauses (1¯3¯ and 4¯6¯) are repeated. (Incidentally, langford′ (12) has 1548 clauses, 417 variables, 3600 literals.)

14. (Answer by M. Heule.) Those clauses sometimes help to focus the search. For example, if we’re trying to color the complete graph Kn with n colors (or pigeons), we don’t want to waste time trying v_2 = 1 when v_1 is already 1.

On the other hand, other instances of SAT often run slower when redundant clauses are present, because more updates to the data structures are needed.

We might also take an opposite approach, and replace (17) by nd clauses that force every color class to be a kernel. (See exercise 21.) Such clauses sometimes speed up a proof of uncolorability.

15. There are N = n(n+1) vertices (j, k) for 0 ≤ j ≤ n and 0 ≤ k < n. If (j, k) = (1, 0) we define (j, k) −−− (n, i) for x ≤ i < n, where x = ⌊n/2⌋. Otherwise we define the following edges: (j, k) −−− (j + 1, k + 1) if j < n and k < n − 1; (j, k) −−− (j + 1, k) if j < n and j ≠ k; (j, k) −−− (j, k + 1) if k < n − 1 and j ≠ k + 1; (j, k) −−− (n, n − 1) if j = 0; (j, k) −−− (n − j, 0) if k < n − 1 and j = k; (j, k) −−− (n + 1 − j, 0) if j > 0 and j = k; (j, k) −−− (n−j, n−j −1) if k = n−1 and 0 < j < k; (j, k) −−− (n + 1−j, n−j)if k = n−1 and 0 < j < n. Finally, (0, 0) −−− (1, 0), and (0, 0) −−− (n, i) for 1 ≤ i ≤ x. That makes a grand total of 3N − 6 edges. (It’s a maximal planar graph; see exercise 7–46.)

16. There’s a unique 4-clique for all n ≥ 3, namely {(0,n − 2), (0,n − 1), (1,n − 1), (n, n−1)}. All other vertices, except (0, 0) and (1, 0), are surrounded by neighbors that form an induced cycle of length 4 or more (usually 6). [See J.-L. Lauriere, Artificial Intelligence 10 (1978), 117.]

17. Let mcgregor (n) be the clauses (15) and (16) for the graph. Add clauses (18) and (19), for symmetric threshold functions to bound the number of variables v_1 for color 1; the kth vertex xk can be specified by the ordering in answer 20. Then if, for instance, we can satisfy those clauses together with the unit clause srN, where N = n(n + 1), we have proved that f(n) < r. Similarly, if we can satisfy them together with s¯rN, we have proved that g(n) ≥ r. Additional unit clauses that specify the colors of the four clique vertices will speed up the computation: Four cases should be run, one with each clique vertex receiving color 1. If all four cases are unsatisfiable, we’ve proved that f(n) ≥ r or g(n) < r, respectively. Binary search with different values of r will identify the optimum.

For speedier g(n), first find a maximum independent set instead of a complete 4-coloring; then notice that the colorings for f(n) already achieve this maximum.

The results turn out to be f(n) = (2, 2, 3, 4, 5, 7, 7, 7, 8, 9, 10, 12, 12, 12) for n = (3, 4,..., 16), and g(n) = (4, 6, 10, 13, 17, 23, 28, 35, 42, 50, 58, 68, 77, 88).

18. Assuming that n ≥ 4, first assign to vertex (j, k) the following “default color”: 1+ (j + k)mod 3 if j ≤ k; 1 + (j + k +1 − n)mod 3 if k < j/2; otherwise 1 + (j + k + 2 − n) mod 3. Then make the following changes to exceptional vertices: Vertex (1, 0) is colored 2 if n mod 6 = 0 or 5, otherwise 3. Vertex (n, n − 1) is colored 4. For k ← 0 up to n − 2, change the color of vertex (n, k) to 4, if its default color matches vertex (0, 0) when k ≤ n/2 or vertex (1, 0) when k > n/2. And make final touchups for 1 ≤ j < n/2, depending again on n mod 6:

Case 0: Give color 4 to vertex (2j, j − 1) and color 1 to vertex (2j + 1,j).

Case 1: Give color 4 to vertex (2j, j) and color 2 to vertex (2j + 1,j).

Case 2: Give color 4 to vertex (2j, j) and color 1 to vertex (2j + 1,j). Also give (n, n − 2) the color 1 and (n − 1,n − 3) the color 4.

Cases 3, 4, 5: Give color 4 to vertex (2j + 1,j).

For example, the coloring for the case n = 10 (found by Bryant) is shown in Fig. A–7(a).

Fig. A–7. Colorings and kernels of McGregor’s graph.

The color distribution is (⌊n^2/3⌋, ⌊n^2/3⌋, ⌊n^2/3⌋, 5k) + ((0, 1, k, −1), (1, k, 1, 0), (−1, k + 1, 1, 2), (0, k, 1, 2), (1, k + 1, 1, 2), (0, 2, k + 1, 3)), for n mod 6 = (0, 1, 2, 3, 4, 5), k = ⌊n/6⌋. Since this construction achieves all of the optimum values for f(n) and g(n), when n ≤ 16, it probably is optimum for all n. Moreover, the value of g(n) agrees with the size of the maximum independent set in all known cases. A further conjecture is that the maximum independent set is unique, whenever n mod 6 = 0 and n > 6.

19. Use the clauses of mcgregor (n), together with (υ1∨υ2∨υ3∨υ¯x)∧(υ1∨υ2∨υ4∨υ¯x)∧(υ1∨υ3∨υ4∨υ¯x)∧(υ2∨υ3∨υ4∨υ¯x) for each vertex, together with clauses from (20) and (21) that require at least r of the vertices vx to be true. Also assign unique colors to the four clique vertices. (One assignment, not four, is sufficient to break symmetry here, because h(n) is a more symmetrical property than f(n) or g(n).) These clauses are satisfiable if and only if h(n) ≥ r. The SAT computation goes faster if we also provide clauses that require each color class to be a kernel (see exercise 21).

The values h(n) = (1, 3, 4, 8, 9, 13) for n = (3, 4,..., 8) are readily obtained in this way. Furthermore, if we extend color class 4 in the construction of answer 18 to a suitable kernel, we find h(9) ≥ 17 and h(10) ≥ 23. The resulting diagram for n = 10, illustrated in Fig. A–7(b), nicely exhibits 2^23 solutions to McGregor’s original coloring problem, all at once.

A good SAT solver also shows that h(9) ≤ 18 and h(10) ≤ 23, thus proving that h(10) = 23. And Armin Biere’s solver proved in 2013 that h(9) = 18, by discovering the surprising solution shown here. (This exercise was inspired by Frank Bernhart, who sent a diagram like Fig. A–7(b) to Martin Gardner in 1975; his diagram achieved 2^21 solutions.)

20. Arrange the vertices (j, k) of answer 15 in the following order v_0, v_1, … : (n, n−1); (0, n − 1), (0, n − 2), … , (0, 0); (1, n − 1), (1, n − 2), … , (1, 1); … ; (n − 2, n − 1), (n − 2, n − 2); (n − 1, n − 2), (n − 2, n − 3), … , (2, 1); (n − 1, n − 1); (2, 0), (3, 1), … , (n, n − 2); (3, 0), (4, 1), … , (n, n − 3); (1, 0); (4, 0), … , (n, n − 4); … ; (n − 1, 0), (n, 1); (n, 0). Then if Vt = {v_0,...,vt−1}, let the “frontier” Ft consist of all vertices ∈ Vt that have at least one neighbor ∉ Vt. We can assume that (v_0,v_1,v_2) are colored (0, 1, 2), because they are part of the 4-clique.

All 4-colorings of Vt that have a given sequence of colors on Ft can be enumerated if we know the corresponding counts for Ft−1. The stated ordering ensures that Ft never will contain more than 2n − 1 elements; in fact, at most 32n−2 sequences of colors are feasible, for any given t. Since 3^18 is less than 400 million, it’s quite feasible to do these incremental calculations. The total (obtained with about 6 gigabytes of memory and after about 500 gigamems of computation) turns out to be 898,431,907,970,211.

This problem is too large to be handled efficiently by BDD methods when n = 10, but BDD calculations for n ≤ 8 can be used to check the algorithm. The frontiers essentially represent level-by-level slices of a QDD for this problem. The 4-coloring counts for 3 ≤ n ≤ 9 are respectively 6, 99, 1814, 107907, 9351764, 2035931737, 847019915170.

21. With one Boolean variable v for every vertex of a graph G, the kernels are characterized by the clauses (i) u¯∨υ¯ whenever u −−− υ; (ii) υ ∨ ∨u−−υ u for all v. Adding to these the clauses for the symmetric threshold function S≤r(x_1,...,xN ), we can find the least r for which all clauses are satisfiable. The graph of Fig. 76 yields satisfiability for r = 17; and one of its 46 kernels of size 17 is shown in Fig. A–7(c).

[BDD methods are slower for this problem; but they enumerate all 520,428,275,749 of the kernels, as well as the generating function 46z^17 + 47180z^18 + … + 317z^34 + 2z^35.]

22. Eight colors are needed. The coloring is “balanced,” with each color used at least thrice.

23. Writing k for xk and kj for sjk, the clauses from (18)–(19) are 1¯112,1¯123,2¯212,2¯223,3¯312,3¯323,4¯412,4¯423,1¯11,2¯21,3¯31, 2¯1¯211,3¯1¯222,4¯1¯233,3¯2¯311,4¯2¯322,5¯2¯333,4¯3¯411,5¯3¯422,6¯3¯433,5¯41,6¯4¯2,7¯4¯3.

Similarly, (20) and (21) define the clauses 7¯16,6¯16,6¯7¯26;5¯15,4¯15,4¯⁢5¯25;3¯14,2¯14,2¯⁢3¯24;1¯13, 6¯311,1¯16¯32,6¯322,1¯26¯33;5¯211,4¯211,5¯222, 4¯222,4¯5¯2123,4¯5¯2123,4¯5¯2213,4¯5¯2224;2¯3¯41,2¯3¯32,2¯3¯23. So this tree-based method apparently needs one more variable and two more clauses when (n, r) = (7, 4). But the next exercise shows that (18) and (19) don’t really win!

24. (a) The clause (b¯12∨b¯r3) appears only if t_3 = r; and t_3 ≤ n/2.

(b) For example, t_3 = min(r, 4) < r when n = 11 and r = 5.

(c) In this case tk is the number of leaves below node k, and the only auxiliary variables that survive pure literal elimination are btkk. We’re left with just n−1 surviving clauses, namely (b¯t2k2k⁢ ∨b¯t2k+12k+1⁢∨btkk) for 1 < k < n, plus (b¯t22∨b¯t33).

(d) If 2k ≤ n ≤ 2k + 2k−1 we have (n′,n″) = (n − 2k−1, 2k−1); on the other hand if 2k + 2k−1 ≤ n ≤ 2k+1 we have (n′,n″) = (2k,n − 2k). (Notice that n″ ≤ n′ ≤ 2n″.)

(e) No pure literals are removed in this completely balanced case (which is the easiest to analyze). We find a(2k, 2k−1) = (k − 1)2k and c(2k, 2k−1) = (2k−2 + k − 1)2k.

(f) One can show that a(n, r) = (r ≤ n″? b(n′, r) + b(n″, r): r ≤ n′? b(n′, n″)+ b(n″, n″): b(n′, n−r) + b(n″, n−r)), where b(1, 1) = 0 and b(n, r) = r + b(n′, min(r, n′)) + b(n″, min(r, n″)) for n ≥ 2. Similarly, c(n, r) = (r ≤ n″? r + f(n′, 0, r) + f(n″, 0, r): r ≤ n′? n″ + f(n′, r − n″, r) + f(n″, 0, n″): n − r + f(n′, r − n″, n′) + f(n″, r − n′, n″)), where f(n,l,r)=Σk=l+1r min(k + 1, n″ + 1, n + 1−k)+(r ≤ n″? r+f(n′, 0, r) + f(n″, 0, r): r ≤ n′? n″ + f(n′, 0, r) + f(n″, 0, n″): r < n? n − r + f(n′, 0, n′) + f(n″, 0, n″): fn′,(n′+l)−˙r,n′)+f(n″,(n″,+l)−˙r,n″)) for n ≥ 2 and f(1, 0, 1) = 0. The desired results follow by induction from these recurrence relations.

Incidentally, ternary branching can give further savings. We can, for example, handle the case n = 6, r = 3 with 17 clauses in the 6 variables b12,b22,b32,b13,b23,b33.

25. From (18) and (19) we obtain 5n − 12 clauses in 2n − 4 variables, with a simple lattice-like structure. But (20) and (21) produce a more complex tree-like pattern, with 2n − 4 variables and with ⌊n/2⌋ nodes covering just two leaves. So we get ⌊n/2⌋ nodes with 3 clauses, n mod 2 nodes with 5 clauses, ⌈n/2⌉ nodes with 7 clauses, and 2 clauses from (21), totalling 5n − 12 as before (assuming that n > 3). In fact, all but n − 2 of the clauses are binary in both cases.

26. Imagine the boundary conditions sj0=1,sjr+1=0,s0k=0, for 1 ≤ j ≤ n − r and 1 ≤ k ≤ r. The clauses say that s1k≤…≤sn−rk and that xj+k⁢ sjk≤sjk+1; so the hint follows by induction on j and k.

Setting j = n − r and k = r + 1 shows that we cannot satisfy the new clauses when x_1 + … + xn ≥ r + 1. Conversely, if we can satisfy F with x_1 + … + xn ≤ r then we can satisfy (18) and (19) by setting sjk←[x1+…+xj+k−1≥k]

27. Argue as in the previous answer, but imagine that b0k=1,br+1⁢ 1=0; prove the hint by induction on j and n − k (beginning with k = n − 1, then k = n − 2, and so on).

28. For example, the clauses for x¯1+…+x¯n≤n−1 when n = 5 are (x1∨s11),(x2∨s¯11∨s12),(x3∨s¯12∨s13),(x4∨s¯13∨s14),(x5∨s¯14). We may assume that n ≥ 4; then the first two clauses can be replaced by (x1∨x2∨s12), and the last two by (xn−1∨xn∨s¯1n−2), yielding n − 2 clauses of length 3 in n − 3 auxiliary variables.

29. We can assume that 1 ≤ r_1 ≤ … ≤ rn = r < n. Sinz’s clauses (18) and (19) actually do the job nicely if we also assert that sjk is false whenever k = ri + 1 and j = i−ri.

30. The clauses now are (s¯jk∨sj+1k),(x¯j+k∨s¯jk∨sjk+1),(sjk∨s¯jk+1),(xj+k∨sjk∨s¯j+1k), hence they define the quantities sjk=[x1+…+xj+k−1≥k]; implicitly s0k=sjr+1=0 and sj0=sn−r+1k=1. The new clauses in answer 23 are 12¯11,23¯11,34¯11,12¯22,23¯22,34¯22,12¯33,23¯33,34¯33; 111¯,212¯,313¯,414¯,211¯12,322¯12,433¯12,544¯12311¯23, 422¯23,533¯23,644¯23431,532,633,734.

With (20) and (21) we can identify b′jk with b¯jk+1−jk, when lk > 1 leaves are below node k. Then bjk is true if and only if the leaves below k have j or more 1s. For example, answer 23 gets the new clauses 726¯,626¯,6726¯;525¯,425¯,4515¯;324¯,224¯,2314¯;133¯, 63¯23,163¯22,63¯12,163¯12,163¯11;42¯24,52¯24,42¯13, 452223,52¯13,452¯122,452¯122,452212,4511;2341,2332,2323.

Furthermore, (20) and (21) can be unified in the same way with the weaker constraints r′ ≤ x_1 + … + xn ≤ r. If we want, say, 2 ≤ x_1 + … + x_7 ≤ 4, we can simply replace the final four clauses of the previous paragraph by 452¯111,2321,2312. Under the conventions of (18) and (19), by contrast, these weaker constraints would generate a comparable number of new clauses, namely 12¯11,12¯22,12¯33,12¯44,12¯55 and 111¯,212¯,322¯12,311¯23,422¯23,411¯34,522¯34,511¯45,622¯45,651,752; but those clauses involve the new variables 14,15,24,25.

31. We can use the constraints on the second line of (10), together with the constraints of exercise 30 that force x_1 + … + xn = r. Then we seek n for which this problem is satisfiable, while the same problem with xn = 0 is not. The following small values can be used to check the calculations:

r=123456789101112131415161718192021222324252627F3(r)=12459111314202426303236404151545863717482849295100F4(r)=12356891013151719212325272830333437404345485053F5(r)=1234678911121314161718192425272829313334363738F6(r)=1234578910121314151718192022232425262932333536

Furthermore, significant speedup is possible if we also make use of previously computed values Ft(1), … , Ft(r−1). For example, when t = 3 and r ≥ 5wemust have xa+1 +…+xa+8 ≤ 4 for 0 ≤ a ≤ n−8, because F_3(5) = 9. These additional subinterval constraints blend beautifully with those of exercise 30, because xa+1 + … + xa+p ≤ q for 0 ≤ a ≤ n − p implies s¯b+p−qk∨sbk−q for 0 ≤ b ≤ n +1 − p + q − r and q < k ≤ r.

We can also take advantage of left-right symmetry by appending the unit clause s¯⌈(n−r)/2⌉⌈r/2⌉ when r is odd; sn/2−r/2+1r/2 when n and r are both even.

Suitable benchmark examples arise when computing, say, F_3(27) or F_4(36). But for large cases, general SAT-based methods do not seem to compete with the best special-purpose backtrack routines. For example, Gavin Theobald and Rodolfo Niborski have obtained the value F_3(41) = 194, which seems well beyond the reach of these ideas.

[See P. Erdös and P. Turán, J. London Math. Soc. (2) 11 (1936), 261–264; errata, 34 (1959), 480; S. S. Wagstaff, Jr., Math. Comp. 26 (1972), 767–771.]

32. Use (15) and (16), and optionally (17), but omit variable vj unless j ∈ L(υ).

33. To double-color a graph with k colors, change (15) to the set of k clauses υ_1 ∨…∨ υj−1 ∨ υj+1 ∨…∨ υk, for 1 ≤ j ≤ k; similarly, (2k) clauses of length k − 2 will yield a triple coloring. Small examples reveal that C2l+1 for l ≥ 2 can be double-colored with five colors: {1, 2} ({3, 4}{5, 1})l− {2,3}{4,5}; furthermore, seven colors suffice for triple coloring when l ≥ 3: {1,2,3} ({4,5,6} {7,1,2})l−2 {3,4,5}{6,7,1}{2,34}{5,6,7}. The following exercise proves that those colorings are in fact optimum.

34. (a) We can obviously find a q-tuple coloring with qχ(G) colors. And McGregor’s graph has a four-clique, hence χ∗.(G) ≥ 4.

(b) Any q-tuple coloring with p colors yields a solution to the fractional exact cover problem, if we let λj=Σi=1p [Sj is the set of vertices colored i]/q. Conversely, the theory of linear equalities tells us that there is always an optimum solution with rational {λ_1, . . . , λN}; such a solution yields a q-tuple coloring when each qλj is an integer.

(c) χ∗(Cn) = χ(Cn) = 2 when n is even; and χ∗(C2l+1) ≤ 2+1/l = n/α(C2l+1), because there’s an l-tuple coloring with n colors as in the previous exercise. Also χ∗(G) ≥ n/α(G) in general: n = Συ Σj λj[υ ∈ Sj] = Σj λj |Sj| ≤ α(G) Σj λj.

(d) For the hint, let S = {υ_1,...,υl} where vertices are sorted by their colors. Since vertex υj belongs to Ci with |Ci| ≥ |{υj,...,υl}|, we have tυj ≤ 1/(l + 1 − j).

So χ(G) ≤ k = Συ tυ = Συ tυ Σj λj[υ ∈ Sj ] = Σj λj Συ t|υ[υ ∈ Sj] ≤ Σj λjHα(G).

[See David S. Johnson, J. Computer and System Sci. 9 (1974), 264–269; L. Lovász, Discrete Math. 13 (1975), 383–390. The concept of fractional covering is due to A. J. W. Hilton, R. Rado, and S. H. Scott, Bull. London Math. Soc. 5 (1973), 302–306.]

35. (a) The double coloring below proves that χ∗(G) ≥ 7/2; and it is optimum because NV and its neighbors induce the wheel W_5. (Notice that χ∗(Wn) = 1 + χ∗(Cn).)

(b) By part (c) of the previous exercise, χ∗(G) ≥ 25/4. Furthermore there is a quadruple coloring with 25 colors:

[Is C5⊠C5¯ the smallest graph for which χ∗(G) < χ(G) − 1?]

36. A few more binary color constraints analogous to (16) yield the corresponding SAT problem. We can also assume that the upper right corner is colored 0, because that region touches n + 4 = 14 others; at least n + 6 colors are needed. The constraints elsewhere aren’t very tight (see exercise 38(b)); thus we readily obtain an optimum radio coloring with n + 6 colors for the McGregor graphs of all orders n > 4, such as the one below. An (n + 7)th color is necessary and sufficient when n = 3 or 4.

37. The 10-coloring shown here is optimum, because Missouri (MO) has degree 8.

38. By looking at solutions for n = 10, say, which can be obtained quickly via Algorithm W (WalkSAT), it’s easy to discover patterns that work in general: (a) Let (x, y) have color (2x + 4y) mod 7. (Seven colors are clearly necessary when n ≥ 3.) (b) Let (x, y, z) have color (2x + 6y) mod 9. (Nine colors are clearly necessary when n ≥ 4.)

39. Let f(n) denote the fewest consecutive colors. SAT solvers readily verify that f(n) = (1, 3, 5, 7, 8, 9) for n = (0, 1, 2, 3, 4, 5). Furthermore we can exploit symmetry to show that f(6) > 10: One can assume that 000000 is colored 0, and that the colors of 000001, …, 100000 are increasing; that leaves only three possibilities for each of the latter. Finally, we can verify that f(6) = 11 by finding a solution that uses only the colors {0, 1, 3, 4, 6, 7, 9, 10}.

But f(7) is known only to be ≥ 11 and ≤ 15.

[L(2, 1) labelings were named by J. R. Griggs and R. K. Yeh, who initiated the theory in SIAM J. Discrete Math. 5 (1992), 586–595. The best known upper bounds, including the fact that f(2k − k − 1) ≤ 2k, were obtained by M. A. Whittlesey, J. P. Georges, and D. W. Mauro, who also solved exercise 38(a); see SIAM J. Discrete Math. 8 (1995), 499–506.]

40. No; the satisfiable cases are z = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 14, 15, 21. [The statement would have been true if we’d also required (xm∨ … ∨x_2) Λ (yn∨…∨y_2).]

41. First there are mn ANDs to form xiyj. A bin that contains t bits initially will generate ⌊t/2⌋ carries for the next bin, using (t − 1)/2 adders. (For example, t = 6 will invoke 2 full adders and one half adder.) The respective values of t for bin [2], bin [3], … , bin [m + n + 1] are (1, 2, 4, 6, … , 2m − 2, 2m − 1, … , 2m − 1, 2m − 2, 2m − 3, … , 5, 3, 1), with n − m occurrences of 2m − 1. That makes a total of mn − m − n full adders and m half adders; altogether we get mn + 2(mn − m − n) + m instances of AND, mn − m − n instances of OR, and 2(mn − m − n) + m instances of XOR.

42. Ternary XOR requires quaternary clauses, but ternary clauses suffice for median:

(t∨u∨v∨x¯)(t∨u¯∨v¯∨x¯)(t¯∨u∨v¯∨x¯)(t¯∨u¯∨v¯∨x¯)(t∨u∨v¯∨x)(t∨u¯∨v∨x)(t¯∨u∨v∨x)(t¯∨u¯∨v¯∨x)(t∨u∨y¯)(t∨v∨y¯)(u∨v∨y¯)(t¯∨u¯∨y)(t¯∨v¯∨y)(u¯∨v¯∨y)

These clauses specify respectively that x ≤ t⊕u⊕υ, x ≥ t⊕u⊕υ, y ≤ 〈tuv〉, y ≥ 〈tuv〉.

43. x = y = 3 works when n = 2, but the cases 3 ≤ n ≤ 7 are unsatisfiable. We can use x = 3(2n−2 + 1), y = 7(2n−3 + 1) for all n ≥ 8. (Such solutions aren’t at all rare. For example, (x, y) = (#C4466223,#E26E7647) is one of 293 instances when n = 32.)

44. First scout the territory quickly by looking at all (N+12)=660 billion cases with at most six zeros in x or y; here N=(3227)+(2732)+…+(3232). This uncovers the remarkable pair x = 2^32 − 2^26 − 2^22 − 2^11 − 2^8 − 2^4 − 1, y = 2^32 − 2^11 + 2^8 − 2^4 + 1, whose product is 2^64 − 2^58 − 2^54 − 2^44 − 2^33 − 2^8 − 1. Now a SAT solver finishes the job by showing that the clauses for 32 × 32 bit multiplication are unsatisfiable in the presence of the further constraint x¯1+⋅⋅⋅+x¯32+y¯1+⋅⋅⋅+y¯32+z¯1+⋅⋅⋅+z¯64≤15. (The LIFO version of the clauses worked much faster than FIFO in the author’s experiments with Algorithm L. Symmetry was broken by separate runs with xk …x_1 = 01k−1, yk …y_1 = 1k.)

45. Use the clauses for xy = z in the factorization problem, with m = ⌊t/2⌋, n = ⌊t/2⌋, and xj = yj for 1 ≤ j ≤ m; append the unit clause (y¯n) if m < n.

46. The two largest, 285000288617375^2 and 301429589329949^2, have 97 bits; the next square binary palindrome, 1178448744881657^2, has 101. [This problem is not easy for SAT solvers; number theory does much better. Indeed, M. Coriand has discovered a nice way to find all n-bit examples by considering only O(2n/4) cases, because the left and right halves of a binary number are nearly forced by the left and right quarters of its square. The first eight square binary palindromes were found by G. J. Simmons, JRM 5 (1972), 11–19; see OEIS sequence A003166 for many further results.]

47. Each wire has a “top” and a “bottom.” There are n + g + 2h tops of wires, and m + 2g + h bottoms of wires. Hence the total number of wires is n + g + 2h = m + 2g + h, and we must have n + h = m + g.

48. The wires compute q^1 ← q, q^2 ← q, x ← p ⊕ q^1, y ← q^2 ⊕ r, z ← x ⊕ y. Let p denote “p stuck at 1” while p¯ denotes “p stuck at 0.” The pattern pqr = 000 detects p, q^1, q^2, r, x, y, z; 001 detects p,q1,q2,r¯,x,y¯,z¯; 010 detects p,q¯1,q¯2,r,x¯,y¯,z; 011 detects p,q¯1,q¯2,r¯,x¯,y,z¯; 100 detects p¯,q1,q2,r,x¯,y,z¯; 101 detectsp¯,q1,q2,r¯,x¯,y¯,z; 110 detects p¯,q¯1,q¯2,r,x,y¯,z; 111 detects p¯,q¯1,q¯2,r¯,x,y,z. Notice that the stuck-at faults for q aren’t detectable (because z = (p ⊕ q) ⊕ (q ⊕ r) = p ⊕ r); but we can detect faults on its clones q^1, q^2. (In Fig. 77 the opposite happens.)

Three patterns such as {100, 010, 001} suffice for all of the detectable faults.

49. One finds, for example, that the faults b32,c¯12,s¯2, and q¯ are detected only by the pattern y_3y_2y_1x_2x_1 = 01111; a¯22,a¯32,b¯32,p¯,c¯22,z¯5 are detected only by 11011 or 11111.

All covering sets can be found by setting up a CNF with 99 positive clauses, one for each detectable fault; for example, the clause for z¯5 is x_27 ∨ x_31, while the clause for x22 is x_4 ∨ x_5 ∨ x_12 ∨ x_13 ∨ x_20 ∨ x_21 ∨ x_28 ∨ x_29. We can find minimum covers from a BDD for these clauses, or by using a SAT solver with additional clauses such as (20) and (21) to limit the number of positive literals. Exactly fourteen sets of five patterns suffice, the most memorable being {01111, 10111, 11011, 11101, 11110}. (Indeed, every minimum set includes at least three of these five patterns.)

50. Primed variables for tarnished wires are x2′,b2′,b3′,s′,z3′,c2′,z4′,z5′. Those wires also have sharped variables x2#,b2#,…,z5#; and we need sharped variables x21#,x23#,x24#,b21#,b22#,b31#,b32#,s1#,s2#c21#,c22# for fanout wires. The primed variables are defined by clauses such as (p¯′∨a3)∧(p¯′∨b′2)∧(p′∨a¯3∨b2′¯), which corresponds to p′ ← a_3 Λ b′2. Those clauses are appended to the 49 clauses listed after (23) in the text. Then there are two clauses (25) for nine of the ten primed-and-sharped variables; however, in the case of x_2 we use the unit clauses (x′2)∧(x¯2) instead, because the variable x2# doesn’t exist. There are five fanout clauses (26), namely (x¯21#∨x23#∨x24#)∧(b¯2#∨b21#∨b22#)∧…∧(c¯2#∨c21#∨c22#). There are eleven clauses (x¯23#∨b2#)∧(x¯24#∨b3#)∧(b¯21#∨s#)∧…∧(b¯32#∨z5#)∧(c¯22#∨z5#) for tarnished inputs to gates. And finally there’s (x21#)∧(z3#∨z4#∨z5).

51. (The complete set of 196 patterns found by the author in 2013 included the inputs (x, y) = (2^32 − 1, 2^31 + 1) and (⌈263/2⌉, ⌈263/2⌉) as well as the two number-theoretic patterns mentioned in the text. Long runs of carries are needed in the products.)

52. (z1,2∨z2,2∨…∨zM,2)∧(z¯i,2∨q¯i,1)∧(z¯i,2∨p¯i,2)∧(z¯i,2∨q¯i,3)∧(z¯i,2∨p¯i,4)∧…∧(z¯i,2∨q¯i,20). The second subscript of z is k in the kth case, 1 ≤ k ≤ P.

53. On the left is the binary expansion of π, and on the right is the binary expansion of e, 20 bits at a time (see Appendix A).

One way to define f(x) for all 20-bit x is to write π/4=Σk=1∞uk/220k and e/4=Σl=1∞υl/220l, where each uk and υl is a 20-bit number. Let k and l be smallest such that x = uk and x = υl. Then f(x) = [k ≤ l].

Equation (27) has actually been contrived to sustain an illusion of magic: Many simple Boolean functions are consistent with the data in Table 2, even if we require four-term DNFs of three literals each. But only two of them, like (27), have the additional property that they actually agree with the definition of f(x) in the previous paragraph for ten more cases, using uk up to k = 22 and υl up to l = 20! One might almost begin to suspect that a SAT solver has discovered a deep new connection between π and e.

54. (a) The function x¯1x9x11x¯18∨x¯6x¯10x¯12∨x¯4x10x¯12 matches all 16 rows of Table 2; but adding the 17th row makes a 3-term DNF impossible.

(b) 21 rows are impossible, but (27) satisfies 20 rows.

(c) x¯1x¯5x¯12x17∨x¯4x8x¯13x¯15∨x¯6x¯9x¯12x16∨x¯6x¯13x¯16x20∨x13x14x¯16 does 28, which is max. (Incidentally, this problem makes no sense for sufficiently large M, because the equation f(x) = 1 probably does not have exactly 2^19 solutions.)

55. Using (28)–(31) with pi,j = 0 for all i and j, and also introducing clauses like (20) and (21) to ensure that qi,1 + … + qi,20 ≤ 3, leads to solutions such as
f(x1,…,x20)=x¯1x¯7x¯8∨x¯2x¯3x¯4∨x¯4x¯13x¯14∨x¯6x¯10x¯12.

(There are no monotone increasing solutions with ≤ 4 terms of any length.)

56. We can define f consistently from only a subset of the variables if and only if no entry on the left agrees with any entry on the right, when restricted to those coordinate positions. For example, the first 10 coordinates do not suffice, because the top entry on the left begins with the same 10 bits as the 14th entry on the right. The first 11 coordinates do suffice (although two entries on the right actually agree in their first 12 bits).

Let the vectors on the left be uk and those on the right be vl, as in answer 53, and form the 256 × 20 matrix whose rows are uk ⊕ υl for 1 ≤ k, l ≤ 16. We can solve the stated problem if and only if we can find five columns for which that matrix isn’t 00000 in any row. This is the classical covering problem (but with rows and columns interchanged): We want to find five columns that cover every row.

In general, such an m × n covering problem corresponds to an instance of SAT with m clauses and n variables xj, where xj means “select column j.” The clause for a particular row is the OR of the xj for each column j in which that row contains 1. For example, in Table 2 we have u_1 ⊕ υ_1 = 01100100111101111000, so the first clause is x_2 ∨ x_3 ∨ x_6 ∨ …∨ x_17. To cover with at most five columns, we add suitable clauses according to (20) and (21); this gives 396 clauses of total length 2894, in 75 variables.

(Of course (205) is only 15504; we don’t need a SAT solver for this simple task! Yet Algorithm D needs only 578 kilomems, and Algorithm C finds an answer in 353 Kμ.)

There are 12 solutions: We can restrict to coordinates xj for j in {1, 4, 15, 17, 20}, {1, 10, 15, 17, 20}, {1, 15, 17, 18, 20}, {4, 6, 7, 10, 12}, {4, 6, 9, 10, 12}, {4, 6, 10, 12, 19}, {4, 10, 12, 15, 19}, {5, 7, 11, 12, 15}, {6, 7, 8, 10, 12}, {6, 8, 9, 10, 12}, {7, 10, 12, 15, 20}, or {8, 15, 17, 18, 20}. (Incidentally, BDD methods show that the number of solutions to the covering problem has the generating function 12z^5 + 994z^6 + 13503z^7 + … + 20z^19 + z^20, counting by the size of the covering set.)

57. Table 2 specifies a partially defined function of 20 Boolean variables, having 2^20−32 “don’t-cares.” Exercise 56 shows how to embed it in a partially defined function of only 5 Boolean variables, in twelve different ways. So we have twelve different truth tables:

And the tenth of these yields f(x)=((x8⁢⊕ (x9∨x10))∨((x6∨x12)⊕ x¯10)) ⊕ x12.

58. These clauses are satisfiable whenever the other clauses are satisfiable (except in the trivial case when f(x) = 0 for all x), because we don’t need to include both xj and x¯j in the same term. Furthermore they reduce the space of possibilities by a factor of (3/4)N . So they seem worthwhile. (On the other hand, their effect on the running time appears to be negligible, at least with respect to Algorithm C in small-scale trials.)

59. f(x)⊕f^(x)=x2x¯3x¯6x¯10x¯12(x¯8∨x8(x13∨x15)) is a function of eight variables that has 7 solutions. Thus the probability is 7/256 = .02734375.

60. A typical example with 32 given values of f(x), chosen randomly, yielded
f^(x1⁢,…, x20)=x4x¯7x¯12∨x¯6x8x¯11x14x20∨x¯9x¯12x18x¯19∨x¯13x¯16x¯17x19,
which of course is way off; it differs from f(x) with probability 102752/2^18 ≈ .39. With 64 training values, however,
f^(x1,…,x20)=x2x¯13x¯15x19∨x¯3x¯9x¯19x¯20∨x¯6x¯10x¯12∨x¯8x10x¯12
comes closer, disagreeing only with probability 404/2^11 ≈ .197.

61. We can add 24 clauses (pa,1∨qa,1∨pa,2∨q¯a,2∨pa,3∨q¯a,3 ∨ … ∨ pb,1∨qb,1∨…∨pc,1∨qc,1∨…∨pd,1∨qd,1 ∨ … ∨ p¯d,10∨qd,10∨…∨pd,20∨qd,20), one for each permutation abcd of {1, 2, 3, 4}; the resulting clauses are satisfiable only by other functions f(x).

But the situation is more complicated in larger examples, because a function can have many equivalent representations as a short DNF. A general scheme, to decide whether the function described by a particular setting pi,j′ and qi,j′ of the ps and qs is unique, would be to add more complicated clauses, which state that pi,j and qi,j give a different solution. Those clauses can be generated by the Tseytin encoding of

∨i=1M  ∧j=1N((p¯i,j∧x¯j)∨(q¯i,j∧xj))⊕∨i=1M∧j=1N((p¯′i,j∧x¯j)∨(q¯′i,j∧xj)).

62. Preliminary experiments by the author, with N = 20 and p = 1/8, seem to indicate that more data points are needed to get convergence by this method, but the SAT solver tends to run about 10 times faster. Thus, locally biased data points appear to be preferable unless the cost of observing the hidden function is relatively large.

Incidentally, the chance that x(k) = x(k−1) was relatively high in these experiments ((7/8)20 ≈ .069); so cases with y(k) = 0 were bypassed.

63. With Tseytin encoding (24), it’s easy to construct 6r + 2n−1 clauses in 2r + 2n−1 variables that are satisfiable if and only if α fails to sort the binary sequence x_1 … xn. For example, the clauses when α = [1 : 2][3 : 4][1 : 3][2 : 4][2 : 3] are (x1∨l¯1)∧(x2∨l¯1)∧(x¯1∨x¯2∨l1)∧(x¯1∨h1)∧(x¯2∨h1)∧(x1∨x2∨h¯1) ∧ … ∧ (l4∨l¯5)∧(h3∨l¯5)∧(l¯4∨h¯3∨l5) ∧ (l¯4∨h5)∧(h¯3∨h5)∧(l4∨h3∨h¯5)∧(g1∨g2∨g3)∧(g¯1∨l3) ∧ (g¯1∨l¯5)∧(g¯2∨l5)∧(g¯2∨h¯5)∧(g¯3∨h5)∧(g¯3∨h¯4). They’re unsatisfiable, so α always sorts properly.

64. Here we reverse the policy of the previous answer, and construct clauses that are satisfiable when they describe a sorting network: Let the variable Ci,jt stand for the existence of comparator [i:j] at time t, for 1 ≤ i < j ≤ n and 1 ≤ t ≤ T . Also, adapting (20) and (21), let variables Bj,kt be defined for 1 ≤ j ≤ n−2 and 1 ≤ k ≤ n, with clauses

(B¯2j,kt∨B¯2j+1,kt)∧(B¯2j,kt∨Bj,kt)∧(B¯2j+1,kt∨Bj,kt)∧(B2j+1,kt∨B2j+1,kt∨B¯j,kt);(∗)

in this formula we substitute {C1,kt,…,Ck−1,k,tCk,k+1t,…,Ck,nt} for the n − 1 “leaf nodes” {Bn−1,kt,…,B2n−3,kt} These clauses prohibit comparators from clashing at time t, and they make B1,kt false if and only if line k remains unused.

If x = x_1 …xn is any binary vector, let y_1 …yn be the result of sorting x (so that (y_1 …yn)2 = 2vx−1). The following clauses F(x) encode the fact that comparators Ci,jt transform x↦y:(C¯i,jt∨V¯x,jt∨Vx,it−1)∧(C¯i,jt∨V¯x,it∨Vx,jt−1)∧(C¯i,jt∨Vx,it∨V¯x,it−1∨V¯x,jt−1)∧ (C¯i,jt∨V¯x,jt∨Vx,it−1∨Vx,jt−1)∧(C¯i,jt∨Vx,jt∨V¯x,it−1)∧(C¯i,jt∨Vx,jt∨V¯x,jt−1)∧(B1,it∨V¯x,it∨Vx,it−1)∧(B1,it∨Vx,jt∨V¯x,it−1), for 1 ≤ i < j ≤ n and 1 ≤ t ≤ T ; here we substitute xj for Vx,j0 and also substitute yj for Vx,jT, thereby simplifying the boundary conditions.

Furthermore, we can remove all variables Vx,it when x has i leading 0s and Vx,jt when x has j trailing 1s, replacing them by 0 and 1 respectively and simplifying further.

Finally, given any sequence α = [i_1 : j_1] … [ir : jr] of initial comparators, T further parallel stages will yield a sorting network if and only if the clauses (∗), together with ∧x F (x) over all x producible by α, are simultaneously satisfiable.

Setting n = 9, α = [1 : 6][2 : 7][3 : 8][4 : 9], and T = 5, we obtain 85768 clauses in 5175 variables, if we leave out the ten vectors x that are already sorted. Algorithm C finds them unsatisfiable after spending roughly 200 megamems; therefore T^(9)>6. (Algorithm L fails spectacularly on these clauses, however.) Setting T ← 6 quickly yields T^(9)≤7 D. Bundala and J. Závodný [LNCS 8370 (2014), 236–247] used this approach to prove in fact that T^(11)=8 and T^(13)=8. Then T. Ehlers and M. Müller extended it [LNCS 9136 (2015), 167– 176], to prove that T^(17)=10, with the surprising optimum network shown here.

65. (a) The goal is to express the transition equation in CNF. There are (84) clauses like (x¯′∨x¯a∨x¯b∨x¯c∨x¯d), one for each choice of four neighbors {a, b, c, d} ⊆ {NW, N, ..., SE}. Also (87) clauses like (x¯′∨xa∨…∨xg), one for each choice of seven. Also (86) like (x¯′∨x∨xa∨…∨xf), for each choice of six. Also (83) like (x′∨x¯a∨x¯b∨x¯c∨xd∨…∨xh), complementing just three. And finally (82) like (x′∨x¯∨x¯a∨x¯b∨xc∨…∨xg) complementing just two and omitting any one of the others. Altogether 70 + 8 + 28 + 56 + 28 = 190 clauses of average length (70 · 5 + 8 · 8 + 28 · 8 + 56 · 9 + 28 · 9)/190 ≈ 7.34.

(b) Here we let x = xij, xNW = x(i−1)(j−1), … , xSE = x(i+1)(j+1), x′=xij′. There are seven classes of auxiliary variables akij,…,gki,j, each of which has two children; the meaning is that the sum of the descendants is ≥ k. We have k ∈ {2, 3, 4} for the a variables, k ∈ {1, 2, 3, 4} for the b and c variables, and k ∈ {1, 2} for d, e, f, g.

The children of aij are b(i|1)j and cij. The children of bij are di(j−(j&2)) and ei(j+(j&2)). The children of cij are fi′j′ and gij, where i′ = i+2 and j′ = (j −1) | 1 if i is odd, otherwise i′ = i and j′ = j −(j &1). The children of dij are x(i−1)(j+1) and xi(j+1). The children of eij are x(i−1)(j−1) and xi(j−1). The children of fij are x(i−1)j and x(i−1)(j+1). Finally, the children of gij are xi″ j″ and xi″ j″, where i′ = i + 1−((i&1) ≪1); and (i″, j″) = (i + 1, j ⊕ 1) if i is odd, otherwise (i″, j″) = (i − 1,j − 1 + ((j &1) ≪ 1)). (OK — this isn’t elegant. But hey, it works!)

If the children of p are q and r, the clauses that define pk are (pk∨q¯k,∨r¯k″) for k′ + k″ = k and (p¯k∨qk′∨rk″) for k′ + k″ = k + 1. In these clauses we omit q¯0 or r¯0; we also omit qm or rm when q or r has fewer than m descendants.

For example, these rules define d135 and d235 by the following six clauses:

(d135∨x¯26),(d135∨x¯36),(d235∨x¯26∨x¯36),(d¯135∨x26∨x36),(d¯235∨x26),(d¯235∨x36).

The variables bkij are defined only when i is odd; dkij and ekij only when i is odd and j mod 4<2;fki,j only when i + j is even. Thus the total number of auxiliary variables per cell (i, j), ignoring small corrections at boundary points, is 3 + 4/2 + 4 + 2/4 + 2/4 + 2/2 + 2 = 13 of types a through g, not 19, because of the sharing; and the total number of clauses per cell to define them is 21 + 16/2 + 16 + 6/4 + 6/4 + 6/2 + 6 = 57, not 77.

Finally we define x′ij from a2ij,a3ij,a4ij, by means of six clauses

(x¯′ij∨a¯4ij),(x¯′ij∨a2ij),(x¯′ij∨xij∨a3ij),(x′ij∨a4ij∨a¯3ij),(x′ij∨x¯ij∨y¯ij),(yij∨a4ij∨a¯2ij),
where yij is another auxiliary variable (introduced only to avoid clauses of size 4).

66. All solutions to (a) can be characterized by a BDD of 8852 nodes, from which we can obtain the generating function 38z^28 + 550z^29 + … + 150z^41 that enumerates them (with a total computation time of only 150 megamems or so). Part (b), however, is best suited to SAT, and X_0 must have at least 38 live cells. Typical answers are

67. Either or at lower left will produce the X_0 of (37) at time 1. But length 22 is impossible: With r = 4 we can verify that all the live cells in X_4 lie in some 3 × 3 subarray. Then with r = 22 we need to rule out only ((39)+(49)+(59))×6=2016 possibilities, one for each viable X_4 within each essentially different 3 × 3 subarray.

68. The author believes that r = 12 is impossible, but his SAT solvers have not yet been able to verify this conjecture. Certainly r = 11 is achievable, because we can continue with the text’s fifth example after prepending

69. Since only 8548 essentially different 4 × 4 bitmaps are possible (see Section 7.2.3), an exhaustive enumeration is no sweat. The small stable patterns arise frequently, so they’ve all been named:

(A glider is also considered to be stable, although it’s not an oscillator.)

70. (a) A cell with three live neighbors in the stator will stay alive.

(b) A 4 × n board doesn’t work; Fig. A–8 shows the 5 × 8 examples.

(c) Again, the smallest-weight solutions with smallest rectangles are shown in Fig. A–8. Oscillators with these rotors are plentiful on larger boards; the first examples of each kind were found respectively by Richard Schroeppel (1970), David Buckingham (1972), Robert Wainwright (1985).

71. Let the variables Xt = xijt characterize the configuration at time t, and suppose we require Xr = X_0. There are q = 8r automorphisms σ that take Xt ↦ X(t + p) mod r τ , where 0 ≤ p < r and τ is one of the eight symmetries of a square grid.

Any global permutation of the N = n^2r variables leads via Theorem E to a canonical form, where we require the solution to be lexicographically less than or equal to the q − 1 solutions that are equivalent to it under automorphisms.

Such lexicographic tests can be enforced by introducing (q−1)(3N −2) new clauses of length ≤ 3, as in (169) — and often greatly simplified using Corollary E.

These additional clauses can significantly speed up a proof of unsatisfiability. On the other hand they can also slow down the search, if a problem has abundant solutions.

In practice it’s usually better to insist only on solutions that are partially canonical, by using only some of the automorphisms and by requiring lexicographic order only on some of the variables.

72. (a) The two 7 × 7s, shown in Fig. A–8, were found by R. Wainwright (trice tongs, 1972) and A. Flammenkamp (jam, 1988).

Fig. A–8. Noteworthy minimal oscillators of periods 2 and 3.

(b) Here the smallest examples are 9 × 13 and 10 × 15; the former has four L-rotors surrounding long stable lines. Readers will also enjoy discovering 10 × 10 and 13 × 13 instances that have full eightfold symmetry. (When encoding such symmetrical problems by using exercise 65(b), we need only compute the transitions between variables xtij for 1 ≤ i ≤ ⌈m/2⌉ and 1 ≤ j ≤ ⌈n/2⌉; every other variable is identical to one of these. However, the auxiliary variables aij, … , gij shouldn’t be coalesced in this way.)

(c,d) Champion heavyweights have small rotors. What a cool four-way snake dance!

73. (a) They don’t have three A neighbors; and they don’t have three B neighbors.

(b) Two examples appear in Fig. A–9, where they are packed as snugly as possible into a 12 × 15 box. This pattern, found by R. W. Gosper about 1971, is called the phoenix, since its living cells repeatedly die and rise again. It is the smallest mobile flipflop; the same idea yields the next smallest (also seen in Fig. A–9), which is 10 × 12.

(c) The nonblank one comes from a 1 × 4 torus; the checkerboard from an 8 × 8. Here are some amazing m × n ways to satisfy the constraints for small m and n:

Notice that infinite one-dimensional examples are implied by several of these motifs; the checkerboard, in fact, can be fabricated by placing diagonals together.

Fig. A–9. Mobile flipflops: An ideal way to tile the floor of a workspace for hackers.

74. Call a cell tainted if it is A with more than one A neighbor or B with more than one B neighbor. Consider the topmost row with a tainted cell, and the leftmost tainted cell in that row. We can assume that this cell is an A, and that its neighbors are S, T, U, V, W, X, Y, Z in the pattern . Three of those eight neighbors are type B, and at least four are type A; several cases need to be considered.

Case 1: W = X = Y = Z = A. Then we must have S = U = V = B and T = 0 (blank), because S, T, U, V aren’t tainted. The three left neighbors of V can’t be type A, since V already has three A neighbors; nor can they be type B, since V isn’t tainted. Hence the tainted X, which must have two B neighbors in the three cells below it, cannot also have two or more A neighbors there.

Case 2: T = A or V = A. If, say, T = A then X = Y = Z = A, and neither V nor W can be type B.

Case 3: S ≠ A, U = A. Then W can’t be type B, and S must be tainted.

Case 4: S = A, U ≠ A. At least one of W, X, Y, Z is B; at least three are A; so exactly three are A. The B can’t be Y, which has four A neighbors. Nor can it be W or Z: That would force V to be blank, hence T = U = B; consequently W = A, Z = B. Since W is tainted, at least two of its right neighbors must be A, contradicting Z = B.

Thus X = B in Case 4. Either T or V is also B, while the other is blank; say T is blank. The three left neighbors of V cannot be A. So they must either all be B (tainting the cell left of S) or all blank. In the latter case the upper neighbors of T must be BBA in that order, since T is blank. But that taints the B above T. A symmetric argument applies if V is blank.

Case 5: S = U = A. Then W ≠ A, and at least two of {X, Y, Z} are A. Now Y = Z = A forces T = V = X = B and W blank, tainting V.

Similarly, X = Y = A forces T = W = Z = B and V blank; this case is more difficult. The three lower neighbors of Y must be AAB, in that order, lest a B be surrounded by four A’s. But then the left neighbors of X are BBB; hence so are the left neighbors of V, tainting the middle one.

Finally, therefore, Case 5 implies that X = Z = A. Either T, V, W, or Y is blank; the other three are B. The blank can’t be T, since T’s upper three neighbors can’t be A. It can’t be W or Y, since V and T aren’t tainted. So T = W = Y = B and V is blank. The left neighbors of S cannot be A, because S isn’t tainted. So the cell left of X must be A. Therefore X must have at least four A neighbors; but that’s impossible, because Y already has three.

Diagonally adjacent A’s are rare. (In fact, they cannot occur in rectangular grids of size 15 × 18 or 16 × 17.) But diligent readers will be able to spot them in Fig. A–9, which exhibits an astonishing variety of different motifs that are possible in large grids.

75. Let the cells alive at times p − 2, p − 1, p be of types X, Y, Z, and consider the topmost row in which a live cell ever appears. Without loss of generality, the leftmost cell in that row is type Z. The cell below that Z can’t be of type Y, because that Y would have three X neighbors and four Y neighbors besides Z and the blank to its left.

Thus the picture must look like , where the three predecessors of Z and the topmost Y are filled in. But there’s no room for the three predecessors of the topmost X.

76. The smallest known example, a 28 × 33 pattern found by Jason Summers in 2012, is illustrated here using the letters {F, A, B}, {B, C, D}, {D, E, F} for cells that are alive when t mod 3 = 0, 1, 2. His ingenious construction leads in particular to an infinite solution based on a 7 × 24 torus. An amazing infinite 7 × 7 toroidal pattern also exists, but little else is yet known.

77. If the first four cells in row 4 of X_0 (and of X_5) contain a, b, c, d, we need a+b ≠ 1, a + b + c ≠ 1, b + c + d ≠ 2. In clause form this becomes a¯∨b,a∨b¯,b∨c¯⁢,c¯∨d,b¯∨c∨d¯.

Similarly, let the last four elements of column 5 be (f, g, h, i); then we want f + g + h ≠ 2, g + h + i ≠ 2, h + i ≠ 2. These conditions simplify to f¯∨g¯,f¯∨h¯,g¯∨ι¯⁢,h¯∨ι¯.

78. The “9^2 phage” in Fig. A–10 is a minimal example.

79. (Solution by T. G. Rokicki.) A tremendous battle flares up, raging wildly on all fronts. When the dust finally settles at time 1900, 11 gliders are escaping the scene (1 going in the original NE direction, 3 going NW, 5 going SW, and 2 going SE), leaving behind 16 blocks, 1 tub, 2 loaves, 3 boats, 4 ships, 8 beehives, 1 pond, 15 blinkers, and 1 toad. (One should really watch this with a suitable applet.)

80. Paydirt is hit on 10 × 10 and 11 × 11 boards, with X_8 = X_9; see Fig. A–10. The minimal example, “symeater19,” has a close relative, “symeater20,” which consists simply of two blocks and two carriers, strategically placed. (The first of these, also called “eater 2,” was discovered by D. Buckingham in the early 1970s; the other by S. Silver in 1998.) They both have the additional ability to eat the glider if it is moved one or two cells to the right of the position shown, or one cell to the left.

It is important to realize that the diagonal track of a glider does not pass through the corners of pixels, bisecting them; the axis of a glider’s symmetry actually passes through the midpoints of pixel edges, thereby cutting off small triangles whose area is 1/8 of a full pixel. Consequently, any eater that is symmetric about a diagonal will eat gliders in two adjacent tracks. The two in Fig. A–10 are exceptional because they’re quadruply effective. Furthermore symeater20 will eat from the opposite direction; and either of its carriers can be swapped to another position next to the blocks.

81. Two eaters make “ssymeater14” (Fig. A–10); and “ssymeater22” is narrower.

82. (a) If X → X′, then x′ij = 1 only if we have Σi′=i−1i+1Σj′=j−1j+1xi′j′≥3.

(b) Use the same inequality, and induction on j.

Fig. A–10. Various examples of minimal still lifes that eat gliders and spaceships.

(c) (Proof of the hint by John Conway, 1970.) In the transitions

we must have in the center of X′; hence we must have at the lower left of X. But then the center of X′ is .

83. Work with (2r + 1 − 2t) × (2r + 1 − 2t) grids xtij centered at cell (i_0, j_0), for 0 ≤ t ≤ r = f(i_0,j_0); and assume that xtij = 0 whenever f(i, j) > t. For example, if (i_0, j_0) = (1, 2), only 14 of the x_3ij can be alive, namely when (i, j) = (−2 .. −1, 2), (−2 .. 0, 1), (−2 .. 1, 0), (−2 .. 2, −1). The case (i_0, j_0) = (1, 2) leads to 5031 readily satisfiable clauses on 1316 variables, including the unit clause x_612, when the state transitions are encoded as in answer 65; all but 106 of those variables are auxiliary.

84. (a) Use a glider, positioned properly with its tip at (0, 0).

(b) Similarly, a spaceship reaches these cells in the minimum possible time.

(c) Consider patterns An = and Bn = of width 2n + 1, illustrated here for n = 3. Then Bj works when j mod 4 ∈ {1, 2}; Aj and Bj−1 work when j mod 4 ∈ {2, 3}; Aj−1 works when j mod 4 ∈ {0, 3}.

(d) The pattern assembles a suitable glider at time 3.

(e) A SAT solver found the pattern shown here, which launches an appropriate spaceship (plus some construction debris that vanishes at t = 5).

[It appears likely that f∗ (i, j) = f(i, j) for all i and j. But the best general result at present, based on space-filling constructions such as Tim Coe’s “Max,” is that f∗(i, j) = f(i, j) + O(1). There’s no known way to prove even the special cases that, say, f∗(j, 2j) = 6j or that f∗(−j, 2j) = 3j for all j ≥ 0.]

85. (a) Let X be a 12 × 12 bitmap. We must show that the clauses T (X, X′) of exercise 65, together with 92 unary clauses x′23, x¯′24, x′25,… from the given pattern, are unsatisfiable. (The pattern is symmetrical; but Life’s rules often produce symmetrical states from unsymmetrical ones.) Thus 2144−8 different conceivable predecessor states need to be ruled out. Fortunately Algorithm C needs fewer than 100 Mμ to do that.

(b) Most states have thousands of predecessors (see the following exercise); so Algorithm C can almost always find one in, say, 500 Kμ. Therefore one can prove, for example, that no 6 × 6 Gardens of Eden exist, by rapidly finding a predecessor for each of the 2^36 patterns. (Only about 2^36/8 patterns actually need to be tried, by symmetry.) Furthermore, if we run through those patterns in Gray code order, changing the polarity of just one assumed unary clause ±xij′ at each step, the mechanism of Algorithm C goes even faster, because it tends to find nearby solutions to nearby problems. Thus thousands of patterns can be satisfied per second, and the task is feasible.

Such an approach is out of the question for 10 × 10 bitmaps, because 2^100 ≫ 2^36. But we can find all 10 × 10 Gardens of Eden for which there is 90°-rotational symmetry, by trying only about 2^25/2 patterns, again using Gray code. Aha: Eight such patterns have no predecessor, and four of them correspond to the given orphan.

[See C. Hartman, M. J. H. Heule, K. Kwekkeboom, and A. Noels, Electronic J. Combinatorics 20, 3 (2013), #P16, 1–19. The existence of Gardens of Eden with respect to many kinds of cellular automata was first proved nonconstructively by E. F. Moore, Proc. Symp. Applied Math. 14 (1962), 17–33.]

86. The 80 cells outside the inner 8 × 8 can be chosen in N = 11,984,516,506,952,898 ways. (A BDD of size 53464 proves this.) So the answer is N/2100−64 ≈ 174,398.

87. Instead of using subscripts t and t + 1, we can write the transition clauses for X → X′ in the form (@⁢∨A0¯∨A0′), etc. Let Alice’s states be {α_1,...,αp} and let Bob’s be {β_1,...,βq}. The clauses (@∨α¯i∨α′i) and (@¯∨β¯i∨β′i) say that your state doesn’t change unless you are bumped. If state α corresponds to the command ‘Maybe go to s’, the clause (@¯∨α¯∨α′∨s′) defines the next possible states after bumping. The analogous clause for ‘Critical, go to s’ or ‘Set υ ← b, go to s’ is simply (@¯∨α¯∨s′); and the latter also generates the clause (@¯∨α¯∨s′) if b=1, (@¯∨α¯∨υ¯′) if b = 0. The command ‘If υ go to s_1, else to s_0’ generates (@¯∨α¯∨υ¯∨s′1)∧(@¯∨α¯∨υ∨s′0). And for each variable υ, if the states whose commands set υ are αi1, …, αih, the clauses

(@¯∨υ∨αi1∨…∨αih∨υ¯′)∧(@¯∨υ¯∨αi1∨…∨αih∨υ′)

encode the fact that υ isn’t changed by other commands.

Bob’s program generates similar clauses — but they use @, not @¯, and β, not α.

Incidentally, when other protocols are considered in place of (40), the initial state X_0 analogous to (41) is constructed by putting Alice and Bob into their smallest possible states, and by setting all shared variables to 0.

88. For example, let all variables be false except A0_0, B0_0, @0, A1_1, B0_1, A1_2, B1_2, A1_3, B2_3, @3, A2_4, B2_4, @4, A3_5, B2_5, l_5, A3_6, B3_6, l_6.

89. No; we can find a counterexample to the corresponding clauses as in the previous exercise: A0_0, B0_0, A0_1, B1_1, A0_2, B2_2, b_2, @2, A1_3, B2_3, b_3, A1_4, B3_4, b_4, A1_5, B4_5, b_5, @5, A2_6, B4_6, a_6, b_6, @6, A5_7, B4_7, a_7, b_7, A5_8, B2_8, a_8, b_8, l_8, A5_9, B5_9, a_9, b_9, l_9.

(This protocol was the author’s original introduction to the fascinating problem of mutual exclusion [see CACM 9 (1966), 321–322, 878], about which Dijkstra had said “Quite a collection of trial solutions have been shown to be incorrect.”)

90. Alice starves in (43) with p = 1 and r = 3 in (47), if she moves to A1 and then Bob remains in B0 whenever he is bumped. The A2 Λ B2 deadlock mentioned in the text for (45) corresponds to (47) with p = 4 and r = 6. And in (46), successive moves to B1, (B2, A1, A2, B3, B1, A4, A5, A0)∞ will starve poor Bob.

91. A cycle (47) with no maybe/critical states for Alice can certainly starve her. Conversely, given (i), (ii), (iii), suppose Alice is in no maybe/critical state when t ≥ t_0; and let t_0 < t_1 < t_2 < … be times with @ti = 1 but with @t = 0 for at least one t between ti and ti+1. Then we must have Xti = Xtj for some i < j, because the number of states is finite. Hence there’s a starvation cycle with p = ti and r = tj.

92. For 0 ≤ i < j ≤ r we want clauses that encode the condition Xi ≠ Xj. Introduce new variables σij for each state σ of Alice or Bob, and vij for each shared variable v. Assert that at least one of these new variables is true. (For the protocol (40) this clause would be (A0ij ∨…∨ A4ij ∨ B0ij ∨…∨ B4ij ∨ lij).) Also assert the binary clauses (σ¯ij∨σi)∧(σ¯ij∨σ¯j) for each σ, and the ternary clauses (υ¯ij∨υi∨υj)∧(υ¯ij∨υ¯i∨υ¯j) for each υ.

The transition clauses can also be streamlined, because we needn’t allow cases where Xt+1 = Xt. Thus, for example, we can omit B0t+1 from the clause (@t∨B0¯t∨B0t+1∨B1t+1) of (42); and we can omit the clause (@t∨B1¯t∨l¯t∨B1t+1) entirely.

[If r is large, encodings with O(r(log r)2) clauses are possible via sorting networks, as suggested by D. Kroening and O. Strichman, LNCS 2575 (2003), 298–309. The most practical scheme, however, seems to be to add the ij constraints one by one as needed; see N. Eén and N. Sörensson, Electronic Notes in Theoretical Computer Science 89 (2003), 543–560.]

93. For the Φ in (50), for example, we can use (x1∨x2∨…∨x16)∧(x¯1∨A0¯′)∧…∧(x¯1∨A6¯′)∧(x¯2∨B0¯′) ∧…∧(x¯2∨B6¯′)∧(x¯3∨A0¯′)∧(x¯3∨a′)⁢∧…∧(x¯16∨B6′)∧(x¯16∨b¯′).

94. (X → X′ → … → X(r)) Λ Φ(X) Λ Φ(X′) ∧ … ∧ Φ(X(r−1)) ∧ ⌝ Φ(X(r)). [This important technique is called “k-induction”; see Mary Sheeran, Satnam Singh, and Gunnar Stålmarck, LNCS 1954 (2000), 108–125. One can, for example, add the clause (A5¯∨B5¯) to (50) and prove the resulting formula Φ by 3-induction.]

95. The critical steps have a = b = 1, by the invariants, so they have no predecessor.

96. The only predecessor of A52∧B52∧a2∧b2∧l¯2 is A51∧B41∧a1∧b1∧l¯1; and the only predecessor of that is A50∧B30∧a0∧b0∧l¯0. The case l_2 is similar.

But without the invariants, we could find arbitrarily long paths to A5r Λ B5r. In fact the longest such simple path has r = 33: Starting with A20∧B20∧a¯0∧b¯0∧l¯0, we could successively bump Alice and Bob into states A3, A5, A6, A0, A1, A2, A3, B3, B4, A5, B3, A6, B4, A0, B3, A1, A2, A3, A5, A6, A0, A1, A2, B4, A3, A5, A6, A0, B5, A1, A2, A3, A5, never repeating a previous state. (Of course all of these states are unreachable from the real X_0, because none of them satisfy Φ.)

97. No. Removing each person’s final step in a path to A6 Λ B6 gives a path to A5 Λ B5.

98. (a) Suppose X_0 → … → Xr = X_0 is impure and Xi = Xj for some 0 ≤ i < j < r. We may assume that i = 0. If either of the two cycles X_0 → … → Xj = X_0 or Xj → … → Xr = Xj is impure, it is shorter.

(b) In those states she would have had to be previously in A0 or A5.

(c) Generate clauses (g¯0),(g¯t∨gt−1∨@t−1),(h¯0),(h¯t∨ht−1∨@t−1¯),(f¯t∨gt), (f¯t∨ht),(f¯t∨α0∨α¯t),(f¯t∨α¯0∨αt),(f¯t∨υ0∨υ¯t)(f¯t∨υ¯0∨υt), for 1 ≤ t ≤ r; and (f_1 ∨ f_2 ∨ … ∨ fr). Here υ runs through all shared variables, and α runs through all states that can occur in a starvation cycle. (For example, Alice’s states with respect to protocol (49) would be restricted to A3 and A4, but Bob’s are unrestricted.)

(d) With exercise 92 we can determine that the longest simple path, using only states that can occur in a starvation cycle for (49), is 15. And the clauses of (c) are unsatisfiable when r = 15 and invariant (50) is used. Thus the only possible starvation cycle is made from two simple pure cycles; and those are easy to rule out.

99. Invariant assertions define the values of a and b at each state. Hence mutual exclusion follows as in exercise 95. For starvation-freedom, we can exclude states A0, A6, A7, A8 from any cycle that starves Alice. But we need also to show that the state A5t ∧ B0t ∧ lt is impossible; otherwise she could starve while Bob is maybe-ing. For that purpose we can add ¬((A6∨A7∨A8)∧(B6∨B7∨B8))∧¬(A8∧l¯) ∧ ¬ (B8∧l)∧¬((A3∨A4∨A5)∧B0∧l)∧¬(A0∧(B3∨B4∨B5)∧l¯) to the invariant Φ(X). The longest simple path through allowable states has length 42; and the clauses of exercise 98(c) are unsatisfiable when r = 42. Notice that Alice and Bob never compete when setting the common variable l, because states A7 and B7 cannot occur together.

(See Dijkstra’s Cooperating Sequential Processes, cited in the text.)

100. Bob is starved by the moves B1, (A1, A2, A3, B2, A4, B3, A0, B4, B1)∞. But an argument similar to the previous answer shows that Alice cannot be.

[The protocol obviously provides mutual exclusion as in exercise 95. It was devised independently in the late 1970s by J. E. Burns and L. Lamport, as a special case of an N-player protocol using only N shared bits; see JACM 33 (1986), 337–339.]

101. The following solution is based on G. L. Peterson’s elegant protocol for N processes in ACM Transactions on Programming Languages and Systems 5 (1983), 56–65:

A0. Maybe go to A1.

B0. Maybe go to B1.

A1. Set a_1 ← 1, go to A2.

B1. Set b_1 ← 1, go to B2.

A2. If b_2 go to A2, else to A3.

B2. If a_1 go to B2, else to B3.

A3. Set a_2 ← 1, go to A4.

B3. Set b_2 ← 1, go to B4.

A4. Set a_1 ← 0, go to A5.

B4. Set b_1 ← 0, go to B5.

A5. If b_1 go to A5, else to A6.

B5. If a_2 go to B5, else to B6.

A6. Set a_1 ← 1, go to A7.

B6. Set b_1 ← 1, go to B7.

A7. If b_1 go to A8, else to A9.

B7. If a_1 go to B8, else to B12.

A8. If b_2 go to A7, else to A9.

B8. If a_2 go to B9, else to B12.

A9. Critical, go to A10.

B9. Set b_1 ← 0, go to B10.

A10. Set a_1 ← 0, go to A11.

B10. If a_1 go to B11, else to B6.

A11. Set a_2 ← 0, go to A0.

B11. If a_2 go to B10, else to B6.

B12. Critical, go to B13.

(Alice and Bob might need an app to help them deal with this.)

B13. Set b_1 ← 0, go to B14.

B14. Set b_2 ← 0, go to B0.

102. The clauses for, say, ‘B5. If a go to B6, else to B7.’ should be (@∨B5¯∨a¯∨α1∨…∨αp∨B6′)∧ (@∨B5¯∨a∨α1∨…∨αp∨B7′)∧(@∨B5¯∨B6′∨B7′), where α_1, …, αp are the states in which Alice sets a.

103. See, for example, any front cover of SICOMP, or of SIAM Review since 1970.

104. Assume that m ≤ n. The case m = n is clearly impossible, because all four corners must be occupied. When m is odd and n = m + k + 1, put m bishops in the first and last columns, then k in the middle columns of the middle row. When m is even and n = m +2k + 1, put m in the first and last columns, and two in the middle rows of columns m/2+2j for 1 ≤ j ≤ k. There’s no solution when m and n are both even, because the maximum number of independent bishops of each color is (m + n − 2)/2. [R. Berghammer, LNCS 6663 (2011), 103–106.]

105. (a) We must have (xij, x′ij) = (1, 0) for t pairs ij, and (0, 1) for t other pairs; otherwise xij = x′ij. Hence there are 2mn−2t solutions.

(b) Use 2mn variables yij,y′ij for 1 ≤ i ≤ m and 1 ≤ j ≤ n, with binary clauses (y¯ij∨y¯′ij), together with m + n +2(m + n − 1) sets of cardinality constraints such as (20) and (21) to enforce the balance condition Σ{yij+y¯′ij|ij⁢∈L|=|L| for each row, column, and diagonal line L.

(c) T (m, n) = 1 when min(m, n) < 4, because only the zero matrix qualifies in such cases. Other values can be enumerated by backtracking, if they are small enough. (The asymptotic behavior is unknown.)

n=45678T(4,n)=37173577T(5,n)=7311093651367T(6,n)=17109877631547607T(7,n)=3536563151076371703883T(8,n)=77136747607170388366291089

(d) Suppose m ≤ n. Any solution with nonzero first row and column has all entries zero except that y_1t = −yt_1 = y(k+1−t)1 = −ykt = yk(k+1−t) = −y(k+1−t)k = ytk = −y1(k+1−t), for some t and k with 1 < t ≤ k/2 ≤ m/2. So the answer is 2 Σk=4m⌊k/2−1⌋(m+1−k)(n+1−k), which simplifies to q(q − 1)(4q(n − q) − 5n + 2q + 3 +(m mod 2)(6n − 8q − 5))/3 when q = ⌊m/2⌋.

[The answer in the case (m, n) = (25, 30) is 36080; hence a random 25 × 30 image will have an average of 36080/256 ≈ 140.9 tomographically equivalent “neighbors” that differ from it in exactly eight pixel positions. Figure 79 has five such neighbors, one of which is shown in answer 111 below.]

(e) We can make all entries nonzero except on the main diagonals (see below). This is optimum, because the diagonal lines for a_1, a_3, … , a_4n−1, b_1, b_3, … , b_4n−1 must each contain a different 0. So the answer is 2n(n − 1). (But the maximum for odd sized boards is unknown; for m = n = (5, 7, 9) it turns out to be (6, 18, 33).)

(f) The smallest counterexamples are 7 × 7 (see above).

106. In an m × n problem we must have 0 ≤ ri ≤ n, 0 ≤ cj ≤ m, and 0 ≤ ad, bd ≤ min{d, m, n, m+n−d}. So the total number B of possibilities, assuming that m ≤ n, is (n + 1)m(m + 1)n((m + 1)! (m +1)n−m m!)2, which is ≈ 3 · 10^197 when (m, n) = (25, 30). Since 2^750/B ≈ 2 · 10^28, we conclude that a “random” 25 × 30 digital tomography problem usually has more than 10^28 solutions. (Of course there are other constraints too; for example, the fact that Σri = Σcj = Σad = Σbd reduces B by at least a factor of (n + 1)(m + 1)2.)

107. (a) (r_1,...,r_6) = (11, 11, 11, 9, 9, 10); (c_1,...,c_13) = (6, 5, 6, 2, 4, 4, 6, 5, 4, 2, 6, 5, 6); (a_1,...,a_6) = (11, 10, 9, 9, 11, 11); (b_1,...,b_12) = (6, 1, 6, 5, 7, 5, 6, 2, 6, 5, 7, 5).

(b) There are two others, namely the following one and its left-right reversal:

[Reference: P. Gerdes, Sipatsi (Maputo: U. Pedagógica, 2009), page 62, pattern #122.]

108. Here are four of the many possibilities:

109. F1. [Initialize.] Find one solution y_1 …yn, or terminate if the problem is unsatisfiable. Then set yn+1 ← 1 and d ← 0.

F2. [Advance d.] Set d to the smallest j > d such that yj = 1.

F3. [Done?] If d > n, terminate with y_1 …yn as the answer.

F4. [Try for smaller.] Try to find a solution with additional unit clauses to force xj = yj for 1 ≤ j < d and xd = 0. If successful, set y_1 …yn ← x_1 … xn.

Return to F2. Even better is to incorporate a similar procedure into the solver itself; see exercise 275.

110. Algorithm B actually gives these directly:

001111111011101111100101111101111011111110111011011111111100101111101111011111100111011111110111 111111111011111111001100111111001111011111111010111111110111101111111001100111110110111101111111

111. This family of problems appears to provide an excellent (though sometimes formidable) series of benchmark tests for SAT solvers. The suggested example has solutions

and several of the entries in (a) were by no means easy. An even more difficult case arises if we base lexicographic order on a rook path that spirals out from the center (thus favoring solutions that are mostly 0 or mostly 1 in the middle):

Here many of the entries have never yet been solved by a SAT solver, as of 2013, although again IP solvers have no great difficulty. In fact, the “lexicographic pure cutting plane” procedure of E. Balas, M. Fischetti, and A. Zanette [Math. Programming A130 (2011), 153–176; A135 (2012), 509–514] turns out to be particularly effective on such problems.

112. Reasonably tight upper and lower bounds would also be interesting.

113. Given an N × N × N contingency problem with binary constraints CJK = X∗JK, RIK = XI∗K, PIJ = XIJ∗, we can construct an equivalent n × n digital tomography problem with n = N^2 + N^3 + N^4 as follows: First construct a four-dimensional tensor YIJKL = X(I⊕L)JK, where I ⊕ L = 1 + (I + L − 1) mod N, and notice that Y∗JKL = YIJK∗ = X∗JK, YI∗KL = X(I⊕L)∗K, YIJ∗L = X(I⊕L)J∗. Then define xij for 1 ≤ i, j ≤ n by the rule xij = YIJKL when i = I − N^2K + N^3L, j = NJ + N^2K + N^3L, otherwise xij = 0. This rule makes sense; for if 1 ≤ I, I′, J, J′, K, K′, L, L′ ≤ N and I − N^2K + N^3L = I′ − N^2K′ + N^3L′ and NJ + N^2K + N^3L = NJ′ + N^2K′ + N^3L′, we have I ≡ I′ (modulo N); hence I = I′ and K ≡ K′; hence K = K′, L = L′, J = J′.

Under this correspondence the marginal sums are ri = YI∗KL when i = I −N^2K + N^3L, cj = Y∗JKL when j = NJ + N^2K + N^3 L, ad = YIJ∗L when d + 1 = I + NJ + 2N^3L, bd = YIJK∗ when d − n = I − NJ − 2N^2K, otherwise zero. [See S. Brunetti, A. Del Lungo, P. Gritzmann, and S. de Vries, Theoretical Comp. Sci. 406 (2008), 63–71.]

114. (a) From x7,23 + x7,24 = x7,23 + x7,24 + x7,25 = x7,24 + x7,25 = 1 we deduce x7,23 = x7,25 = 0 and x7,24 = 1, revealing n7,23 = n7,25 = 5. Now x6,23 + x6,24 = + x6,24 = x4,25 + x4,24 + x5,24 + x6,24 + x6,25 = 1; hence x4,24 = x5,24 = 0, revealing n4,24 = n5,24 = 2. So x6,23 = x6,25 = 0, and the rest is easy.

(b) Let yi,j mean “cell (i, j) has been probed safely, revealing ni,j.” Consider the clauses C obtained by appending y¯i,j to each clause of the symmetric function [Σi′=i−1i+1Σj′=j−1j+1xi′,j′=ni,j], for all i, j with xi,j = 0. Also include (x¯i,j∨y¯i,j), as well as clauses for the symmetric function SN (x) if we’re told the total number N of mines.

Given any subset F of mine-free cells, the clauses CF = C ∧ Λ {yi,j | (i, j) ∈ F} are satisfiable precisely by the configurations of mines that are consistent with the data {ni,j | (i, j) ∈ F}. Therefore cell (i, j) is safe if and only if CF ∧ xi,j is unsatisfiable.

A simple modification of Algorithm C can be used to “grow” F until no further safe cells can be added: Given a solution to CF for which neither xi,j nor x¯i,j was obtained at root level (level 0), we can try to find a “flipped” solution by using the complemented value as the decision at level 1. Such a solution will be found if and only if the flipped value is consistent; otherwise the unflipped value will have been forced at level 0. By changing default polarities we can favor solutions that flip many variables at once. Whenever a literal x¯i,j is newly deduced at root level, we can force yi,j to be true, thus adding (i, j) to F . We reach an impasse when a set of solutions has been obtained for CF that covers both settings of every unforced xi,j.

For problem (i) we start with F = {(1, 1)}, etc. Case (iv) by itself uncovers only 56 cells in the lower right corner. The other results, each obtained in < 6 Gμ, are:

Notice that the Cheshire cat’s famous smile defies logic and requires much guesswork!

[For aspects of Minesweeper that are NP-complete and coNP-complete, see Kaye, Scott, Stege, and van Rooij, Math. Intelligencer 22, 2 (2000), 9–15; 33, 4 (2011), 5–17.]

115. Several thousand runs of the algorithm in the previous exercise, given that the total number of mines is 10, indicate success probabilities .490 ± .007, .414 ± .004, .279 ± .003, when the first guess is respectively in a corner, in the center of an edge, or in the center.

116. The smallest is the “clock” in answer 69(b). Other noteworthy possibilities are

as well as the “phoenix” in Fig. A–9.

117. (a) Set x_0 = xn+1 = 0, and let (a, b, c) be respectively the number of occurrences of (01, 10, 11) as a substring of x_0x_1 …xn + 1. Then a + c = b + c = νx and c = ν(2)x; hence a = b = νx − ν(2)x is the number of runs.

(b) In this case the complete binary tree will have only n−1 leaves, corresponding to {x_1x_2,...,xn−1xn}; therefore we want to replace n by n − 1 in (20) and (21).

The clauses of (20) remain unchanged unless tk ≤ 3. When tk = 2 they become (x¯2k−n+1∨x¯2k−n+2∨b1k)∧(x¯2k−n+2∨x¯2k−n+3∨b1k)∧(x¯2k−n+1∨x¯2k−n+2∨x¯2k+n+3∨b2k). When tk = 3 we have 2k = n−1, and they become (b¯12k∨b1k)∧(x¯∨x¯2∨b1k)∧(b¯22k∨b2k)∧(b¯12k∨x¯1∨x¯2∨b2k)∧(b¯22k∨x¯1∨x¯2∨b3k).

The clauses of (21) remain unchanged except in simple cases when n ≤ 3.

(c) Now the leaves represent xixi+1¯=x¯i∨x¯i+1. So we change (20), when tk = 2, to (x2k−n+1∨b1k)∧(x2k−n+2∨b1k)∧(x2k−n+3∨b1k)∧(x2k−n+2∨b2k)∧(x2k−n+1∨x2k−n+3∨b2k). And there are eight clauses when tk=3:(b¯12k∨b1k)∧(x1∨b1k)∧(x2∨b1k)∧(b¯22k∨b2k) ∧ (b¯12k∨x1∨b2k)∧(b¯12k∨x2∨b2k)∧(b¯22k∨x1∨b3k)∧(b¯22k∨x2∨b3k).

118. Let pi,j = [the pixel in row i and column j should be covered], and introduce variables hi,j when pi,j = pi,j+1 = 1, υi,j when pi,j = pi+1,j = 1. The clauses are (i) (hi,j ∨ hi,j−1 ∨ υi,j ∨ υi−1,j), whenever pi,j = 1, omitting variables that don’t exist; (ii) (h¯i,j∨h¯i,j−1),(h¯i,j∨υ¯i,j),(h¯i,j∨υ¯i−1,j),(h¯i,j,−1∨υ¯i,j),(h¯i,j,−1∨υ¯i−1,j),(υ¯i,j∨υ¯i−1,j) whenever pi,j = 1, omitting clauses whose variables don’t both exist; and (iii) (hi,j ∨ hi+1,j ∨ υi,j ∨ υi,j+1), whenever pi,j + pi,j+1 + pi+1,j+1 ≥3, omitting variables that don’t exist. (The example has 10527 clauses in 2874 variables, but it’s quickly solved.)

119. ThereÙs symmetry between l and l¯, also between l and 10-l; so we need consider only l = (1, 2, 3, 4, 5), with respectively (4, 4, 6, 6, 8) occurrences. The smallest result is F/5={123,234,678,789,246,468,147,369,1¯⁢2¯⁢3¯,2¯⁢3¯⁢4¯,3¯⁢4¯⁢, 4¯⁢6¯,6¯⁢7¯,6¯⁢7¯⁢8¯,7¯⁢8¯⁢9¯,1¯⁢3¯,2¯⁢4¯⁢6¯,3¯⁢7¯,4¯⁢6¯⁢8¯,7¯⁢9¯,1¯⁢4¯⁢7¯,2¯⁢8¯⁢,3¯⁢6¯⁢9¯,1¯⁢9¯⁢}.

120. True.

121. The main point of interest is that an empty clause is typically discovered in the midst of step A3; partial backtracking must be done when taking back the changes that were made before this interruption.

A3. [Remove l¯.] Set p ← F(l¯) (which is F(l ⊕ 1), see (57)). While p ≥ 2n + 2, set j ← C(p), i ← SIZE(j), and if i > 1 set SIZE(j) ← i − 1, p ← F(p). But if i = 1, interrupt that loop and set p ← B(p); then while p ≥ 2n + 2, set j ← C(p), i ← SIZE(j), SIZE(j) ← i + 1, p ← B(p); and finally go to A5.

A4. [Deactivate l’s clauses.] Set p ← F(l). While p ≥ 2n + 2, set j ← C(p), i ← START(j), p ← F(p), and for i ≤ s < i + SIZE(j) − 1 set q ← F(s), r ← B(s), B(q) ← r, F(r) ← q, and C(L(s)) ← C(L(s)) − 1. Then set a ← a − C(l), d ← d + 1, and return to A2.

A7. [Reactivate l’s clauses.] Set a ← a + C(l) and p ← B(l). While p ≥ 2n +2, set j ← C(p), i ← START(j), p ← B(p), and for i ≤ s < i + SIZE(j) − 1 set q ← F(s), r ← B(s), B(q) ← F(r) ← s, and C(L(s)) ← C(L(s)) + 1. (The links dance a little here.)

A8. [Unremove l¯.] Set p ← F(l¯). While p ≥ 2n + 2, set j ← C(p), i ← SIZE(j), SIZE(j) ← i + 1, p ← F(p). Then go to A5.

122. Pure literals are problematic when we want all solutions, so we don’t take advantage of them here. Indeed, things get simpler; only the move codes 1 and 2 are needed.

A1∗. [Initialize.] Set d ← 1.

A2∗. [Visit or choose.] If d > n, visit the solution defined by m_1 …mn and go to A6∗. Otherwise set l ← 2d + 1 and md ← 1.

A3∗. [Remove l¯.] Delete l¯ from all active clauses; but go to A5∗ if that would make a clause empty.

A4∗. [Deactivate l’s clauses.] Suppress all clauses that contain l. Then set d ← d + 1 and return to A2∗.

A5∗. [Try again.] If md = 1, set md ← 2, l ← 2d, and go to A3∗.

A6∗. [Backtrack.] Terminate if d = 1. Otherwise set d ← d − 1 and l ← 2d + (md & 1).

A7∗. [Reactivate l’s clauses.] Unsuppress all clauses that contain l.

A8∗. [Unremove l¯.] Reinstate l¯ in all the active clauses that contain it. Then go back to A5∗.

It’s no longer necessary to update the values C(k) for k < 2n + 2 in steps A4∗ and A7∗.

123. For example, we might have
p=01234567891011121314151617181920L(p)=397875653438286964742
and START(j) = 21 − 3j for 0 ≤ j ≤ 7; W_2 = 3, W_3 = 7, W_4 = 4, W_5 = 0, W_6 = 5, W_7 = 1, W_8 = 6, W_9 = 2. Also LINK(j) = 0 for 1 ≤ j ≤ 7 in this case.

124. Set j←Wl¯. While j ≠ 0, a literal other than l¯ should be watched in clause j, so we do the following: Set i ← START(j), i′ ← START(j − 1), j′ ← LINK(j), k ← i + 1. While k < i′, set l′ ← L(k); if l′ isn’t false (that is, if |l′| > d or l′ + m|l′| is even, see (57)), set L(i) ← l′, L(k) ← l¯, LINK(j) ← Wl′, Wl′ ← j, j ← j′, and exit the loop on k; otherwise set k ← k + 1 and continue that loop. If k reaches i′, however, we cannot stop watching l¯; so we set Wl¯←j, exit the loop on j, and go on to step B5.

125. Change steps B2 and B4 to be like A2∗ and A4∗ in answer 122.

126. Starting with active ring (6 9 7 8), the unit clause 9 will be found (because 9 appears before 8); the clause 9¯⁢3¯⁢6¯ will become 6¯⁢3¯⁢9¯; the active ring will become (7 8 6).

127. Before: 11414545; after: 1142. (And then 11425, etc.)

128. Active ring x_1x_2x_3x_4 Units Choice Changed clauses

(1234)−−−−1¯213¯(234)0−−−2¯3¯⁢12,324¯(34) 0⁢ 0−−3¯3¯4¯⁢23,431(4)0⁢ 0 0 −4,4¯Backtrack(34)0⁢−−−23¯⁢2¯⁢4(34)0⁢ 1 −−4¯4¯341,1¯⁢42(3)0⁢  1⁢− 03,3¯Backtrack(43)− −−−141¯⁢2,2¯⁢1¯⁢3(243)1−−−2¯(43)1 0 −−44324¯,14¯⁢2¯(3)1 0 −13,3¯Backtrack(43)1−−−231¯⁢2¯(43)1 1−−3342¯⁢3¯,1⁢ 3¯⁢2,4¯⁢3¯⁢1(4)1 1 1−4,4¯Backtrack

129. Set j ← Wl, then do the following steps while j ≠ 0: (i) Set p ← START(j) + 1; (ii) if p = START(j − 1), return 1; (iii) if L(p) is false (that is, if x|L(p)| = L(p) & 1), set p ← p + 1 and repeat (ii); (iv) set j ← LINK(j). If j becomes zero, return 0.

130. Set l ← 2k + b, j ← Wl, Wl ← 0, and do the following steps while j ≠ 0: (i) Set j′ ← LINK(j), i ← START(j), p ← i + 1; (ii) while L(p) is false, set p ← p + 1 (see answer 129; this loop will end before p = START(j − 1)); (iii) set l′ ← L(p), L(p) ← l, L(i) ← l′; (iv) set p ← Wl′ and q ← Wl¯′ , and go to (vi) if p ≠ 0 or q ≠ 0 or x|l′| ≥ 0; (v) if t = 0, set t ← h ← |l′| and NEXT(t) ← h, otherwise set NEXT(|l′|) ← h, h ← |l′|, NEXT(t) ← h (thus inserting |l′| = l′ ≫ 1 into the ring as its new head); (vi) set LINK(j) ← p, Wl′ ← j (thus inserting j into the watch list of l′); (vii) set j ← j′.

[The tricky part here is to remember that t can be zero in step (v).]

131. For example, the author tried selecting a variable xk for which s_2k · s_2k+1 is maximum, where sl is the length of l’s watch list plus ε, and the parameter ε was 0.1. This reduced the runtime for waerden (3, 10; 97) to 139.8 gigamems, with 8.6 mega-nodes. Less dramatic effects occurred with langford (13): 56.2 gigamems, with 10.8 meganodes, versus 99.0 gigamems if the minimum s_2k · s_2k+1 was chosen instead.

132. The unsatisfiable clauses (x¯1∨x2),(x1∨x¯2),(x¯3∨x4),(x3∨x¯4), …, (x¯2n−1∨x2n),(x2n−1∨x¯2n),(x¯2n−1∨x¯2n),(x2n−1∨x2n) cause it to investigate all 2n settings of x_1, x_3, … , x_2n−1 before encountering a contradiction and repeatedly backtracking.

(Incidentally, the successive move codes make a pretty pattern. If the stated clauses are ordered randomly, the algorithm runs significantly faster, but it still apparently needs nonpolynomial time. What is the growth rate?)

133. (a) Optimum backtrack trees for n-variable SAT problems can be calculated with Θ(n3n) time and Θ(3n) space by considering all 3n partial assignments, “bottom up.” In this 9-variable problem we obtain a tree with 67 nodes (the minimum) if we branch first on x_3 and x_5, then on x_6 if x_3 ≠ x_5; unit clauses arise at all other nodes.

(b) Similarly, the worst tree turns out to have 471 nodes. But if we require the algorithm to branch on a unit clause whenever possible, the worst size is 187. (Branch first on x_1, then x_4, then x_7; avoid opportunities for unit clauses.)

134. Let each BIMP list be accessed by ADDR, BSIZE, CAP, and K fields, where ADDR is the starting address in MEM of a block that’s able to store CAP items, and CAP = 2K; ADDR is a multiple of CAP, and BSIZE is the number of items currently in use. Initially CAP = 4, K = 2, BSIZE = 0, and ADDR is a convenient multiple of 4. The 2n BIMP tables therefore occupy 8n slots initially. If MEM has room for 2M items, those tables can be allocated so that the doubly linked lists AVAIL[k] initially contain ak = (0 or 1) available blocks of size 2k for each k, where 2M − 8n = (aM−1 …a_1a_0)2.

Resizing is necessary when BSIZE = CAP and we need to increase BSIZE. Set a ← ADDR, k ← K, CAP ← 2k+ 1, and let b ← a ⊕ 2k be the address of a’s buddy. If b is a free block of size 2k, we’re in luck: We remove b from AVAIL[k]; then if a &2k =0, nothing needs to be done, otherwise we copy BSIZE items from a to b and set ADDR ← b.

In the unlucky case when b is either reserved or free of size < 2k, we set p to the address of the first block in AVAIL[k′], where AVAIL[t] is empty for k < t < k′ (or we panic if MEM’s capacity is exceeded). After removing p from AVAIL[k′], we split off new free blocks of sizes 2k+1, … , 2k′−1 if k′ > k + 1. Finally we copy BSIZE items from block a to block p, set ADDR ← p, and put a into AVAIL[k]. (We needn’t try to “collapse” a with its buddy, since the buddy isn’t free.)

135. They’re the complements of the literals in BIMP(l¯).

136. Before, {(1, 2), (4, 2), (4, 5), (5, 1), (5, 7), (6, 9)}; after, {(1, 2), (4, 2), (6, 9)}.

137. If p in a TIMP list points to the pair (u, v), let’s write u = U(p) and v = V(p).

(a) Set N ← n − G, x ← VAR[N], j ← INX[X], VAR[j] ← x, INX[x] ← j, VAR[N] ← X, INX[X] ← N. Then do the following for l = 2X and l = 2X + 1, and for all p in TIMP(l): u ← U(p), υ ← V(p), p′ ← LINK(p), p″ ← LINK(p′); s ← TSIZE(u¯)−1, TSIZE(u¯) ← s, t ← pair s of TIMP(u¯); if p′ ≠ t, swap pairs by setting u′ ← U(t), v′ ← V(t), q ← LINK(t), q′ ← LINK(q), LINK(q′) ← p′, LINK(p) ← t, U(p′) ← u′, V(p′) ← v′, LINK(p′) ← q, U(t) ← υ, V(t) ← l¯, LINK(t) ← p″, p′ ← t. Then set s ← TSIZE(υ¯)−1, TSIZE(υ¯) ← s, t ← pair s of TIMP(υ¯); if p″ ≠ t, swap pairs by setting u′ ← U(t), υ′ ← V(t), q ← LINK(t), q′ ← LINK(q), LINK(q′) ← p″, LINK(p′) ← t, U(p″) ← u′, V(p″) ← v′, LINK(p″) ← q, U(t) ← l¯, V(t) ← u, LINK(t) ← p.

Notice that we do not make the current pairs of TIMP(l) inactive. They won’t be accessed by the algorithm until it needs to undo the swaps just made.

(b) In VAR and in each TIMP list, the active entries appear first. The inactive entries follow, in the same order as they were swapped out, because inactive entries never participate in swaps. Therefore we can reactivate the most-recently-swapped-out entry by simply increasing the count of active entries. We must, however, be careful to do this “virtual unswapping” in precisely the reverse order from which we did the swapping.

Thus, for l = 2X + 1 and l = 2X, and for all p in TIMP(l), proceeding in the reverse order from (a), we set u ← U(p), v ← V(p), TSIZE(υ¯) ← TSIZE(υ¯) + 1, and TSIZE(υ¯) ← TSIZE(υ¯) +1.

(The number N of free variables increases implicitly, because N + E = n in step L12. Thus nothing needs to be done to VAR or INX. These efficient techniques based on swapping are examples of “sparse-set representations”; see 7.2.2–(16) thru (23).)

138. Because υ¯ ∈ BIMP(u¯), (62) will be used to make u nearly true. That loop will also make v nearly true, because v ∈ BIMP(u) is equivalent to u¯ ∈ BIMP(υ¯).

139. Introduce a new variable BSTAMP analogous to ISTAMP, and a new field BST(l) analogous to IST(l) in the data for each literal l. At the beginning of step L9, set BSTAMP ← BSTAMP + 1, then set BST(l) ← BSTAMP for l=u¯ and all l ∈ BIMP(u¯). Now, if BST(υ¯) = BSTAMP and BST(υ) ≠ = BSTAMP, do the following for all w ∈ BIMP(υ): If ωis fixed in context NT (it must be fixed true, since ω¯ implies υ¯), do nothing. Otherwise if BST( ω¯) = BSTAMP, perform (62) with l ← u and exit the loop on w (because u¯ implies both ω and ω¯). Otherwise, if BST(ω) ≠ = BSTAMP, append ω to BIMP(u¯) and u to BIMP( ω¯). (Of course (63) must be invoked when needed.)

Then increase BSTAMP again, and do the same thing with u and υ reversed.

140. Unfortunately, no: We might have Ω(n) changes to BSIZE on each of Ω(n) levels of the search tree. However, the ISTACK will never have more entries than the total number of cells in all BIMP tables (namely 2M in answer 134).

141. Suppose ISTAMP ← (ISTAMP + 1)mod2e in step L5. If ISTAMP = 0 after that operation, we can safely set ISTAMP ← 1 and IST(l) ← 0 for 2 ≤ l ≤ 2n + 1. (A similar remark applies to BSTAMP and BST(l) in answer 139.)

142. (The following operations, performed after BRANCH[d] is set in step L2, will also output ‘|’ to mark levels of the search where no decision was made.) Set BACKL[d] ← F , r ← k ← 0, and do the following while k < d: While r < BACKF[k], output ‘6+(Rr &1)’ and set r ← r+1. If BRANCH[k] < 0, output ‘|’; otherwise output ‘2BRANCH[k]+(Rr&1)’ and set r ← r + 1. While r < BACKL[k +1], output ‘4 + (Rr & 1)’ and set r ← r +1. Then set k ← k +1.

143. The following solution treats KINX and KSIZE as the unmodified algorithm treats TIMP and TSIZE. It deals in a somewhat more subtle way with CINX and CSIZE: If clause c originally had size k, and if j of its literals have become false while none have yet become true, CSIZE(c) will be k − j, but the nonfalse literals will not necessarily appear at the beginning of list CINX(c). As soon as j reaches k −2, or one of the literals becomes true, clause c becomes inactive and it disappears from the KINX tables of all free literals. The algorithm won’t look at CINX(c) or CSIZE(c) again until it unfixes the literal that deactivated c. Thus a big clause is inactive if and only if it has been satisfied (contains a true literal) or has become binary (has at most two nonfalse literals).

We need to modify only the three steps that involve TIMP. The modified step L1, call it L1′, inputs the big clauses in a straightforward way.

Step L7′ removes the formerly free variable X from the data structures by first deactivating all of the active big clauses that contain L: For each of the KSIZE(L) numbers c in KINX(L), and for each of the CSIZE(c) free literals u in CINX(c), we swap c out of u’s clause list as follows: Set s ← KSIZE(u) − 1, KSIZE(u) ← s; find t ≤ s with KINX(u)[t] = c; if t ≠ s set KINX(u)[t] ← KINX(u)[s] and KINX(u)[s] ← c. [Heuristic: If the number of free literals remaining in c is small compared to c’s original size, for example if say 15 or 20 original literals have become false, the remaining nonfalse literals can usefully be swapped into the first CSIZE(c) positions of CINX(c) when c is being deactivated. The author’s experimental implementation does this when CSIZE(c) is at most θ times the original size, where the parameter θ is normally 25/64.]

Then step L7′ updates clauses for which L has become false: For each of the KSIZE(L¯) numbers c in KINX(L¯), set s ← CSIZE(c) − 1 and CSIZE(c) ← s; if s = 2, find the two free literals (u, υ)in CINX(c), swap them into the first positions of that list, put them on a temporary stack, and swap c out of the clause lists of u and υ as above.

Finally, step L7′ does step L8′ = L8 for all (u, v) on the temporary stack. [The maximum size of that stack will be the maximum of KSIZE(l) over all l, after step L1′; so we allocate memory for that stack as part of step L1′.]

In step L12′ we set L ← RE, X ← |L|, and reactivate the clauses that involve X as follows: For each of the KSIZE(L¯) numbers c in KINX(L¯), proceeding in reverse order from the order used in L7′, set s ← CSIZE(c), CSIZE(c) ← s + 1; if s = 2, swap c back into the clause lists of υ and u, where u = CINX(c)[0] and υ = CINX(c)[1]. For each of the KSIZE(L) numbers c in KINX(L), and for each of the CSIZE(c) free literals u in CINX(c), again proceeding in reverse order from the order used in L7′, swap c back into the clause list of u. The latter operation simply increases KSIZE(u) by 1.

ParamILS advises changing α from 3.5 to 0.001(!) in (195).

144. False; h′(l) = 0.1 if and only if the complement, l¯, doesn’t appear in any clause.

145. By symmetry we know that h(l) = h(l¯) = h(10 − l) for 1 ≤ l ≤ 9 at depth 0, and the BIMP tables are empty. The first five rounds of refinement respectively give (h(1),...,h(5)) ≈ (4.10, 4.10, 6.10, 6.10, 8.10), (5.01, 4.59, 6.84, 6.84, 7.98), (4.80, 4.58, 6.57, 6.57, 8.32), (4.88, 4.54, 6.72, 6.67, 8.06), and (4.85, 4.56, 6.63, 6.62, 8.23), slowly converging to the limiting values

(4.85810213, 4.55160111, 6.66761920, 6.63699698, 8.16778057).

But when d = 1, the successively refined values of (h(1),h(1¯),…,h(4),h(4¯)) are erratic and divergent: (2.10, 9.10, 3.10, 6.60, 3.10, 13.60, 4.10, 11.10), (5.63, 3.37, 9.24, 2.57, 5.48, 5.67, 8.37, 4.87), (1.42, 10.00, 2.31, 10.42, 1.28, 17.69, 1.94, 16.07), (8.12, 1.43, 12.42, 1.30, 7.51, 2.41, 12.02, 1.81), (0.32, 14.72, 0.42, 16.06, 0.30, 26.64, 0.43, 24.84).

They eventually oscillate between limits that favor either positive or negative literals:

(0.1017, 20.6819, 0.1027, 21.6597, 0.1021, 32.0422, 0.1030, 33.0200) and

(8.0187, 0.1712, 11.9781, 0.1361, 11.9781, 0.2071, 15.9374, 0.1718).

[Equations (64) and (65), which were inspired by survey propagation, first appeared in unpublished work of S. Mijnders, B. de Wilde, and M. J. H. Heule in 2010. The calculations above indicate that we needn’t take h(l) too seriously, although it does seem to yield good results in practice. The author’s implementation also sets h′(l) ← Θ if the right-hand side of (65) exceeds a threshold parameter Θ, which is 20.0 by default.]

146. Good results have been obtained with the simple formula h(l) = ε + KSIZE(l¯) + Σu∈BIMP(l), u free KSIZE(u¯), which estimates the potential number of big-clause reductions that occur when l becomes true. The parameter ε is typically set to 0.001.

147. ∞, 600, 60, 30, 30.

148. If a problem is easy, we don’t care if we solve it in 2 seconds or in .000002 seconds. On the other hand if a problem is so difficult that it can be solved only by looking ahead more than we can accomplish in a reasonable time, we might as well face the fact that we won’t solve it anyway. There’s no point in looking ahead at 60 variables when d = 60, because we won’t be able to deal with more than 2^50 or so nodes in any reasonable search tree.

149. The idea is to maintain a binary string SIG(x) for each variable x, representing the highest node of the search tree in which x has participated. Let bj = [BRANCH[j] =1], and set σ ← b_0 …bd−1 at the beginning of step L2, σ ← b_0 …bd at the beginning of step L4. Then x will be a participant in step X3 if and only if SIG(x) is a prefix of σ.

We update SIG(x) when x = |u| or x = |v| in step L9, by setting SIG(x) ← σ unless SIG(x) is a prefix of σ. The initial value of SIG(x) is chosen so that it is never a prefix of any possible σ.

(Notice that SIG(x) needn’t change when backtracking. In practice we can safely maintain only the first 32 bits of σ and of each string SIG(x), together with their exact lengths, because lookahead computations need not be precise. In answer 143, updates occur not in step L9 but in step L7′; they are done for all literals u≠L¯ that appear in any big clause containing L¯ that is being shortened for the first time.)

150. Asserting 7 at level 22 will also 22fix 1¯, because of the clause 1¯⁢4¯⁢7¯. Then 1¯ will 22fix 3 and 9, which will 22fix 2¯ and 6¯, then 8¯; and clause 258 becomes false. Therefore 7¯ becomes proto true; and (62) makes 3, 6, 9 all proto true, contradicting 3¯⁢6¯⁢9¯.

151. For example, one such arrangement is
l:28¯931¯67¯4¯476¯13¯9¯82¯ol:4210146168122226182820243230.

[Digraphs that are obtainable in this way are called “partial orderings of dimension ≤ 2,” or permutation posets. We’ve actually seen them in exercise 5.1.1–11, where the set of arcs was represented as a set of inversions. Permutation posets have many nice properties, which we shall study in Section 7.4.2. For example, if we reverse the order of the list and complement the offsets, we reverse the directions on the arrows. All but two of the 238 connected partially ordered sets on six elements are permutation posets. Unfortunately, however, permutation posets don’t work well with lookahead when they aren’t also forests. For example, after 10fixing ‘9’ and its consequences, we would want to remove those literals from the R stack when 14fixing ‘3’; see (71). But then we’d want them back when 6fixing ‘1¯’.]

152. A single clause such as ‘12’ or ‘123’ would be an example, except that the autarky test in step X9 would solve the problem before we ever get to step X3. The clauses {123¯,12¯⁢3,1¯⁢23¯,1¯⁢2¯⁢3,245,34¯⁢5¯} do, however, work: Level 0 branches on x_1, and level 1 discovers an autarky with x_2 and x_3 both true but returns l = 0. Then level 2 finds all clauses satisfied, although both of the free variables x_4 and x_5 are newbies.

[Indeed, the absence of free participants means that the fixed-true literals form an autarky. If TSIZE(l) is nonzero for any free literal l, some clause is unsatisfied. Otherwise all clauses are satisfied unless some free l has an unfixed literal l′ ∈ BIMP(l).]

153. Make the CAND array into a heap, with an element x of least rating r(x) at the top. (See Section 5.2.3; but start indices at 0, with r(CAND[k]) ≤ min(r(CAND[2k+1]), r(CAND[2k+2])).) Then, while C > Cmax, delete the top of the heap (namely CAND[0]).

154. The child → parent relations in the subforest will be d→c→a, b→a,c¯→d¯, and either a¯→b¯ or a¯→c¯. Here’s one suitable sequence, using the latter:

preorderb¯accdd¯c¯a¯2.postorder210486161412

155. First construct the dependency graph on the 2C candidate literals, by extracting a subset of arcs from the BIMP tables. (This computation needn’t be exact, because we’re only calculating heuristics; an upper bound can be placed on the number of arcs considered, so that we don’t spend too much time here. However, it is important to have the arc u → υ if and only if υ¯→u¯ is also present.)

Then apply Tarjan’s algorithm [see Section 7.4.1.2, or SGB pages 512–519]. If a strong component contains both l and l¯ for some l, terminate with a contradiction. Otherwise, if a strong component contains more than one literal, choose a representative l with maximum h(l); the other literals of that component regard l as their parent. Be careful to ensure that l is a representative if and only if l¯ is also a representative.

The result will be a sequence of candidate literals l_1l_2 …lS in topological order, with li → lj only if i > j. Compute the “height” of each lj, namely the length of the longest path from lj to a sink. Then every literal of height h > 0 has a predecessor of height h − 1, and we let one such predecessor be its parent in the subforest. Every literal of height 0 (a sink) has a null parent. Traversal of this subforest in double order (exercise 2.3.1–18) now makes it easy to build the LL table in preorder while filling the LO table in postorder.

156. If l¯ doesn’t appear in any clause of F , then A = {l} is clearly an autarky.

157. Well, any satisfying assignment is an autarky. But more to the point is the autarky {1, 2} for F={12¯⁢3,1¯⁢24,3¯⁢4¯}.

158. BIMP(l) and TIMP(l) will be empty, so w will be zero when Algorithm X looks ahead on l. Thus l will be forced true, at depth d = 0. (But pure literals that arise in subproblems for d > 0 won’t be detected unless they’re among the preselected candidates.)

159. (a) False (consider A = {1}, F = {1, 2, 1¯2}); but true if we assume that F | A is computed as a multiset (so that F | A would be {2, 2} ⊆ F in that example).

(b) True: Suppose A = A′ ∩ A″, ∪ A′ ∩ A″ = ø, and A″ or ø A¯″ touches C ∈ F | A′. Then C ∩ A′ = ø and C ∩ C′ ∈ F, where C′⊆ A¯′. Since A or A touches C ∪ C′, some a ∈ C ∪ C′ is in A; hence a ∈ A″.

160. (a) If the gray clauses are satisfiable, let all black literals be true. [Notice, incidentally, that the suggested example coloring works like a charm in (7).]

(b) Given any set A of strictly distinct literals, color l black if l ∈ A, white if l¯ ∈ A, otherwise gray. Then A is an autarky if and only if condition (a) holds.

[E. A. Hirsch, Journal of Automated Reasoning 24 (2000), 397–420.]

161. (a) If F′ is satisfiable, so is F . If F is satisfiable with at least one blue literal false, so is F′. If F is satisfiable with all the blue literals true, make all the black literals true (but keep gray literals unchanged). Then F′ is satisfied, because every clause of F′ that contains a black or blue literal is true, hence every clause that contains a white literal is true; the remaining clauses, whose literals are only orange and gray, each contain at least one true gray literal. [The black-and-blue condition is equivalent to saying that A is a conditional autarky, namely an autarky of F | L. Tseytin’s notion of “extended resolution” is a special case, because the literals of A and L need not appear in F. See S. Jeannicot, L. Oxusoff, and A. Rauzy, Revue d’intelligence artificielle 2 (1988), 41–60, Section 6; O. Kullmann, Theoretical Comp. Sci. 223 (1999), 1–72, Sections 3, 4, and 14.]

(b) Without affecting satisfiability, we are allowed to add or delete any clause C=(a∨l¯1∨…∨l¯q) for which all clauses containing ā also contain l_1 or … or lq. (Such a clause is said to be “blocked” with respect to a, because C produces nothing but tautologies when it is resolved with clauses that contain ā.)

(c) Without affecting satisfiability, we are allowed to add or delete any or all of the clauses (l¯∨a1),…,(l¯∨ap), if A is an autarky of F | l; that is, we can do this if A is almost an autarky, in the sense that every clause that touches Ā but not A contains l.

(d) Without affecting satisfiability, we are allowed to add or delete the clause (l¯ ∨ a) whenever every clause that contains ā also contains l.

162. Construct a “blocking digraph” with l′ ↪ l when every clause that contains literal l¯ also contains l′. (If l is a pure literal, we’ll have l′ ↪ l for all l′; this case can be handled separately. Otherwise all in-degrees will be less than k in a kSAT problem, and the blocking digraph can be constructed in O(k^2m) steps if there are m clauses.)

(a) Then (l ∨ l′) is a blocked binary clause if and only if l¯ ↪ l′ or l¯′ ↪ l. (Hence we’re allowed in such cases to add both l¯ ↪ l′ and l¯′ ↪ l to the dependency digraph.)

(b) Also A = {a, a′} is an autarky if and only if a ↪ a′ ↪ a. (Moreover, any strong component {a_1,...,at} with t > 1 is an autarky of size t.)

163. Consider the recurrence relations Tn = 1 + max(Tn−1, Tn−2, 2Un−1), Un = 1 + max(Tn−1 ,Tn−2 ,Un−1 +Vn−1), Vn = 1 + Un−1 for n > 0, with T−1 = T_0 = U_0 = V_0 = 0. We can prove that Tn, Un, Vn are upper bounds on the step counts, where Un refers to cases where F is known to have a nonternary clause, and Vn refers to cases when s = 1 and R2 was entered from R3: The terms Tn−1 and Tn−2 represent autarky reductions in step R2; otherwise the recursive call in R3 costs Un−1, not Tn−1, because at least one clause contains l¯s. We also have Vn = 1 + Un−1, not 1 + Tn−1, because the preceding step R3 either had a clause containing l_2 not l_1 or a clause containing l¯1 not l¯2.

Fibonacci numbers provide the solution: Tn = 2Fn+2 −3 + [n =0], Un = Fn+3 −2, Vn = Fn+2 −1. [Algorithm R is a simplification of a procedure devised by B. Monien and E. Speckenmeyer, Discrete Applied Mathematics 10 (1985), 287–295, who introduced the term “autarky” in that paper. A Stanford student, Juan Bulnes, had discovered a Fibonacci-bounded algorithm for 3SAT already in 1976; his method was, however, unattractive, because it also required Ω(ϕn) space.]

164. If k < 3, Tn = n is an upper bound; so we may assume that k ≥ 3. Let Un = 1 + max(Tn−1, Tn−2, Un−1 + Vn−1,1,..., Un−1 + Vn−1, k−2), Vn,1 = 1 + Un−1, and Vn,s = 1 + max(Un−1, Tn−2, Un−1 + Vn−1, s−1) for s > 1, where Vn,s refers to an entry at R2 from R3. The use of Un−1 in the formula for Vn,s is justified, because the previous R3 either had a clause containing ls+1 not ls or one containing l¯s not l¯s+1. One can show by induction that Vn,s = s + Un−1 + … + Un−s, Un = Vn,k−1; and Tn = Un + Un−k+1 = 2Un−1+1 if n ≥ k. For example, the running time when k = 4 is bounded by Tribonacci numbers, whose growth rate 1.83929n comes from the root of x^3 = x^2+x+1.

165. Clause 1¯⁢3¯⁢4¯ in the example tells us that 1, 3, 4 ∉ A. Then 13⁢6¯ implies 6 ∉ A. But A = {2, 5} works, so it is maximum. There always is a maximum (not just maximal) positive autarky, because the union of positive autarkies is a positive autarky.

Each clause (υ1∨…∨υs∨υ¯s+1∨…∨υ¯s+t) of F , where the υ’s are positive, tells us that υ_1 ∉ A and … and υs ∉ A implies υs+j ∉ A, for 1 ≤ j ≤ t. Thus it essentially generates t Horn clauses, whose core is the set of all positive literals not in any positive autarky. A simple variant of Algorithm 7.1.1C will find this core in linear time; namely, we can modify steps C1 and C5 in order to get t Horn clauses from a single clause of F .

[By complementing a subset of variables, and prohibiting another subset, we can find the largest autarky A contained in any given set of strictly distinct literals. This exercise is due to unpublished work of O. Kullmann, V. W. Marek, and M. Truszczyński.]

166. Assume first that PARENT(l_0) = Λ, so that H(l_0) = 0 at the beginning of X9 (see X6). Since l_0 = LL[j] is not fixed in context T , we have RF = l_0 by (62). And A = {RF, RF+1,...,RE−1} is an autarky, because no clause touched by A or Ā is entirely false or contains two unfixed literals. Thus we’re allowed to force l_0 true (which is what “do step X12 with l ← l_0” means).

On the other hand if w = 0 and PARENT(l_0) = p, so that H(l_0) = H(p) > 0 in X6, the set A = {RF ,...,RE−1} is an autarky with respect to the clauses of F | p. Hence the additional clause (l0∨p¯) doesn’t make the clauses any less satisfiable, by the black and blue principle. (Notice that (l¯0∨p) is already a known clause; so in this case l_0 is essentially being made equal to its parent.)

[The author’s implementation therefore goes further and includes the step

VAL[|l0|] ← VAL[|p|]⊕((l0⊕p)&1),(∗)

which promotes the truth degree of l_0 to that of p. This step violates the invariant relation (71), but Algorithm X doesn’t rely on (71).]

167. If a literal l is fixed in context T during the lookahead, it is implied by l_0. In step X11 we have a case where l is also implied by l¯0; hence we’re allowed to force its truth, if l isn’t already proto true. In step X6, l¯0 is implied by l_0, so l_0 must be false.

168. The following method works well in march: Terminate happily if F = n. (At this point in Algorithm L, F is the number of fixed variables, all of which are really true or really false.) Otherwise find l ∈ {LL[0],..., LL[S − 1]} with l mod2 = 0 and maximum (H(l) + .1)(H(l + 1) + .1). If l is fixed, set l ← 0. (In that case, Algorithm X found at least one forced literal, although U is now zero; we want to do another lookahead before branching again.) Otherwise, if H(l) > H(l + 1), set l ← l + 1. (A subproblem that is less reduced will tend to be more satisfiable.)

169. When a and b are positive, the function f(x) = e−ax + e−bx − 1 is convex and decreasing, and it has the unique root ln τ (a, b). Newton’s method for solving this equation refines an approximation x by computing x′ = x + f(x)/(ae−ax + be−bx). Notice that x is less than the root if and only if f(x) > 0; furthermore f(x) > 0 implies f(x′) > 0, because f(x′) > f(x)+(x′ − x)f′(x) when f is convex. In particular we have f(1/(a + b)) > 0, because f(0) = 1 and 0′ = 1/(a + b), and we can proceed as follows:

K1. [Initialize.] Set j ← k ← 1, x ← 1/(a_1 + b_1).

K2. [Done?] (At this point (aj, bj) is the best of (a_1, b_1), … ,(ak, bk), and e−ajx + e−bjx ≥ 1.) Terminate if k = s. Otherwise set k ← k + 1, x′ ← 1/(ak + bk).

K3. [Find α, β.] If x′ < x, swap j ↔ k and x ↔ x′. Then set α ← e−ajx′ and β ← e−bjx′. Go to K2 if α + β ≤ 1.

K4. [Newtonize.] Set x′ ← x′ + (α + β −)/(aj α + bj β), α′ ← e−akx′, β ← e−bkx′, x′ ← x′ + (α′ + β′ − 1)/(akα′ + bkβ′), and return to K3.

(The floating point calculations should satisfy eu ≤ eυ and u + ω ≤ υ + ω when u < υ.)

170. If the problem is unsatisfiable, Tarjan’s algorithm discovers l and l¯ in the same strong component. If it’s satisfiable, Algorithm X finds autarkies (because ω is always zero), thus forcing the value of all literals at depth 0.

171. It prevents double-looking on the same literal twice at the same search tree node.

172. When Algorithm Y concludes normally, we’ll have T = BASE+LO[j], even though BASE has changed. This relation is assumed to be invariant in Algorithm X.

173. The run reported in the text, using nonoptimized parameters (see exercise 513), did 29,194,670 double-looks (that is, executions of step Y2), and exited 23,245,231 times to X13 in step Y8 (thus successfully forcing l_0 false in about 80% of those cases). Disabling Algorithm Y (i) increased the running time from 0.68 teramems to 1.13 teramems, with 24.3 million nodes. Disabling wraparound (ii) increased the time to 0.85 teramems, with 13.3 million nodes. Setting Y = 1, which disabled wraparound only in Algorithm Y, yielded 0.72 teramems, 11.3 meganodes. (Incidentally, the loops of Algorithm X wrapped around 40% of the time in the regular run, with a mean of 0.62 and maximum of 12; those of Algorithm Y had 20% wraparound, with a mean of 0.25; the maximum Y = 8 was reached only 28 times.) Disabling the lookahead forest (iii) gave surprisingly good results: 0.70 teramems, 8.5 meganodes; there were fewer nodes [hence a more discriminating lookahead], but more time spent per node because of duplicated effort, although strong components were not computed. (Structured problems that have numerous binary clauses tend to generate more helpful forests than random 3SAT problems do.) Disabling compensation resolvents (iv) made very little difference: 0.70 teramems, 9.9 meganodes. But disabling windfalls (v) raised the cost to 0.89 teramems and 13.5 meganodes. And branching on a random l ∈ LL (vi) made the running time soar to 40.20 teramems, with 594.7 meganodes. Finally, disabling Algorithm X altogether (vii) was a disaster, leading to an estimated run time of well over 10^20 mems.

The weaker heuristics of exercise 175 yield 3.09 teramems and 35.9 meganodes.

174. Setting Y to a huge value such as PT will never get to step Y2. (But for (ii), (iii), … , (vii) one must change the programs, not the parameters as they stand.)

175. Precompute the weights, by setting K_2 = 1 and Ks ← γKs−1 + .01, for s between 3 and the maximum clause size. (The extra .01 keeps this from being zero.) The third line of (72) must change to “take account of c for all c in KINX(L¯),” where that means “set s ← CSIZE(c) − 1; if s ≥ 2, set CSIZE(c) ← s and ω ← ω + Ks; otherwise if all literals of c are fixed false, set a flag; otherwise if some literal u of c isn’t fixed (there will be just one), put it on a temporary stack.” Before performing the last line of (72), go to CONFLICT if the flag is set; otherwise, for each unfixed u on the temporary stack, set Wi ← u and i ← i+1 and perform (62) with l ← u; go to CONFLICT if some u on the temporary stack is fixed false. (A “windfall” in this more general setting is a clause for which all but one literal has been fixed false as a consequence of l_0 being fixed true.)

Of course those changes to CSIZE need to be undone; a simulated false literal that has been “virtually” removed from a clause must be virtually put back. Fortunately, the invariant relation (71) makes this task fairly easy: We set G ← F in step X5, and insert the following restoration loop at the very beginning of (72): “While G > F , set u ← RG−1; stop if u is fixed in context T ; otherwise set G ← G − 1, and increase CSIZE(c) by 1 for all c ∈ KINX(u¯).” The restoration loop should also be performed, with T ← NT, just before terminating Algorithm X in steps X7 or X13.

[The additional step (∗) in answer 166 can’t be used, because (71) is now crucial.]

Algorithm Y should change in essentially the same way as Algorithm X.

[See O. Kullmann, Report CSR 23-2002 (Swansea: Univ. of Wales, 2002), §4.2.]

176. (a) aj −−− aj+1, aj −−− bj, aj −−− bj+1, bj −−− cj, bj −−− dj, cj −−− dj, cj −−− ej, dj −−− fj, ej −−− dj+1, ej −−− fj+1, fj −−− cj+1, fj −−− ej+1.

(b) Let (tj, uj, υj, ωj, aj, bj, cj, dj, ej, fj) have colors (1, 2, 1, 1, 1, 2, 1, 3, 3, 2) when j is even, (2, 1, 2, 2, 3, 2, 3, 1, 1, 2) when j is odd. The lower bounds are obvious.

(c) Vertices aj, ej, fj can’t all have the same color, because bj, cj, dj have distinct colors. Let αj denote the colors of ajejfj. Then αj = 112 implies αj+1 = 332 or 233; αj = 121 implies αj+1 = 233 or 323; αj = 211 implies αj+1 = 323 or 332; αj = 123 implies αj+1 = 213 or 321. Since α_1 = αq+1, the colors of α_1 must be distinct, and we can assume that α_1 = 123. But then αj will be an odd permutation whenever j is even.

[See Rufus Isaacs, AMM 82 (1975), 233–234. Unpublished notes of E. Grinberg show that he had independently investigated the graph J_5 in 1972.]

177. There are 20 independent subsets of Vj = {aj, bj, cj, dj, ej, fj} when q > 1; eight of them contain none of {bj, cj, dj} while four contain bj. Let A be a 20 × 20 transition matrix, which indicates when R ∪ C is independent for each independent subset R ⊆ Vj and C ⊆ Vj+1. Then Iq is trace(Aq); and the first eight values are 8, 126, 1052, 11170, 112828, 1159416, 11869768, 121668290. The characteristic polynomial of A, x^12(x^2 − 2x − 1)(x^2 + 2x − 1)(x^4 − 8x^3 − 25x^2 + 20x + 1), has nonzero roots ±1±2⁢  and ≈ −2.91, −0.05, + 0.71, + 10.25; hence Iq = Θ(rq), where r ≈ 10.24811166 is the dominant root. Note: The number of kernels of L(Jq) is respectively 2, 32, 140, 536, 2957, 14336, 70093, 348872, for 1 ≤ q ≤ 8, and its growth rate is ≈ 4.93q.

178. With the first ordering, the top 18k levels of the search tree essentially represent all of the ways to 3-color the subgraph {aj, bj, cj, dj, ej, fj | 1 ≤ j ≤ k}; and there are Θ(2k) ways to do that, by answer 176. But with the second ordering, the top 6kq levels essentially represent all of the independent sets of the graph; and there are Ω(10.2k) of those, by answer 177.

Empirically, Algorithm B needs respectively 1.54 megamems, 1.57 gigamems, and 1.61 teramems to prove unsatisfiability when q = 9, 19, and 29, using the first ordering; but it needs 158 gigamems already for q = 5 with the second! Additional clauses, which require color classes to be kernels (see answer 14), reduce that time to 492 megamems.

Algorithm D does badly on this sequence of problems: When q = 19, it consumes 37.6 gigamems, even with the “good” ordering. And when q = 29, its cyclic method of working somehow transforms the good ordering into a bad ordering on many of the variables at depths 200 or more. It shows no sign of being anywhere near completion even after spending a petamem on that problem!

Algorithm L, which is insensitive to the ordering, needs 2.42 megamems, 2.01 gigamems, and 1.73 teramems when q = 9, 19, and 29. Thus it appears to take Θ(2q) steps, and to be slightly slower than Algorithm B as q grows, although exercise 232 shows that a clairvoyant lookahead procedure could theoretically do much better.

Algorithm C triumphs here, as shown in Fig. 92.

179. This is a straightforward exact cover problem. If we classify the solutions according to how many asterisks occur in each coordinate, it turns out that exactly (10, 240, 180, 360, 720, 480, 1440, 270, 200, 480) of them are respectively of type (00088, 00268, 00448, 00466, 02248, 02266, 02446, 04444, 22228, 22246).

By complementation, we see that 4380 choices of 8 clauses are unsatisfiable; hence q8=1−4380/(808)=1−4380/28987537150≈0.9999998.

180. With N variables yj, one for each possible clause Cj, the function f(y_1,...,yN ) = [∧{Cj | yj = 1} is satisfiable ] is ∨x fx(y), where fx(y) = [x satisfies ∧{Cj | yi = 1}] is simply ∧{y¯i| x makes Cj false}. For instance if k = 2 and n = 3, and if C_1, C_7, C_11 are the clauses (x1∨x2),(x1∨x¯3),(x2∨x¯3), then f001(y1,…,y12)=y¯1∧y¯7∧y¯11.

Each function fx has a very simple BDD, but of course the OR of 2n of them will not be simple. This problem is an excellent example where no natural ordering of the clause variables is evident, but the method of sifting is able to reduce the BDD size substantially. In fact, the clauses for k = 3 and n = 4 can be ordered cleverly so that the corresponding 32-variable BDD for satisfiability has only 1362 nodes! The author’s best result for k = 3 and n = 5, however, was a BDD of size 2,155,458. The coefficients of its generating function (exercise 7.1.4–25) are the desired numbers Qm.

The largest such count, Q_35 = 3,449,494,339,791,376,514,416, is so enormous that we could not hope to enumerate the relevant sets of 35 clauses by backtracking.

181. The previous exercise essentially computed the generating function Σm Qmzm; now we want the double generating function Σl,m Tl,mωlzm, where Tl,m is the number of ways to choose m different k-clauses in such a way that these clauses are satisfied by exactly l vectors x_1 …xn. To do this, instead of taking the OR of the simple functions fx, we compute the BDD base that contains all of the symmetric Boolean functions Sl(f_0...0,...,f_1...1) for 0 ≤ l ≤ 2n, as follows (see exercise 7.1.4–49): Consider the subscript x to be a binary integer, so that the functions are fx for 0 ≤ < 2n. Start with Sl = 0 for −1 ≤ l ≤ 2n, except that S_0 = 1. Then do the following for x = 0, … , 2n − 1 (in that order): Set Sl = fx? Sl−1: Sl for l = x +1, … , 0 (in that order).

After this computation, the generating function for Sl will be Σm Tl,mzm. In the author’s experiments, the sifting algorithm found an ordering of the 80 clauses for k = 3 and n = 5 so that only about 6 million nodes were needed when x had reached 24; afterwards, however, sifting took too long, so it was turned off. The final BDD base had approximately 87 million nodes, with many nodes shared between the individual functions Sl. The total running time was about 22 gigamems.

182. T_0 = 32 and T_1 = 28 and Tm = 0 for 71 ≤ m ≤ 80. Otherwise min Tm < max Tm.

183. Let tm = Pr(Tm = 1), and suppose that we obtain clauses one by one until reaching an unsatisfiable set. The fact that tm gets reasonably large suggests that we probably have accumulated a uniquely satisfiable set just before stopping. (That probability is 2−kN Σm tm/(N −m), which turns out to be ≈ 0.8853 when k = 3 and n = 5.)

However, except for the fact that both Figs. 85 and 86 are bell-shaped curves with roughly the same tendency to be relatively large or small at particular values of m, there is apparently no strong mathematical connection. The probabilities in Fig. 86 sum to 1; but the sum of probabilities in Fig. 85 has no obvious significance.

When n is large, uniquely satisfiable sets are encountered only rarely. The final set before stopping a.s. has at most f(n) solutions, for certain functions f; but how fast does the smallest such f grow? [See D. J. Aldous, J. Theoretical Probability 4 (1991), 197–211, for related ideas.]

184. The probability q^m is Q^m/Nm, where Q^m counts the choices (C_1,...,Cm) for which C_1 ∧…∧ Cm is satisfiable. The number of such choices that involve t distinct clauses is t! {mt} times Qt, because {mt} enumerates set partitions; see Eq. 3.3.2–(5).

185. q^m=Σt=0N{mt}t! qt(Nt)/Nm≥qm⁢  Σt=0N{mt}t!(Nt)/Nm=qm.

186. ΣmΣt{mt}t! qt(Nt)N−m can be summed on m, since Σm{mt} N−m=1/(N−t)t¯ by Eq. 1.2.9–(28). Similarly, the derivative of 1.2.9–(28) shows that Σm⁢ m{mt}N−m=(N/(N−1)+…+N/(N−t))/(N−t)t¯.

187. In this special case, qm = [0 ≤ m<N ] and pm = [m = N ]; hence Sn,n = N = 2n (and the variance is zero). By (78), we also have S^n,n=NHN; indeed, the coupon collector’s test (exercise 3.3.2–8) is an equivalent way to view this situation.

188. Now qm=2mnm¯/(2n)m¯. It follows by (78) that S^1,n=Σm=0n⁢ 2mnm¯/(2n−1)m¯, because N = 2n. The identity 2mnm¯/(2n−1)m¯=2qm−qm+1 yields the surprising fact that S^1,n=(2q0−q1)+(2q1−q2)+…1+S1,n; and we also have S^1,n−1=2n2n−1S1,n−1. Hence, by induction, we obtain the (even more surprising) closed forms

S1,n=4n/(2nn),S^1,n=4n/(2nn)+1.

So random 1SAT problems become unsatisfiable after πn+O(1) clauses, on average.

189. With the autosifting method in the author’s experimental BDD implementation, the number of BDD nodes, given a sequence of m distinct clauses when k = 3 and n = 50, increased past 1000 when m increased from 1 to about 30, and it tended to peak at about 500,000 when m was slightly more than 100. Then the typical BDD size fell to about 50,000 when m = 150, and to only about 500 when m = 200.

BDD methods break down when n is too large, but when they apply we can count the total number of solutions remaining after m steps. In the author’s tests with k = 3, n = 50, and m = 200, this number varied from about 25 to about 2000.

190. For example, S_1(x_1,...,xn) can’t be expressed in (n − 1)CNF: All clauses of length n − 1 that are implied by S_1(x_1,...,xn) are also implied by S≤1(x_1,...,xn).

191. Let f(x_0,...,x2n−1) = 1 if and only if x_0 …x2n−1 is the truth table of a Boolean function of n variables that is expressible in kCNF. This function f is the conjunction of 2n constraints c(t), for 0 ≤ t = (t_0 …t_2n−1)2 < 2n, where c(t) is the following condition: If xt = 0, then ∨{xy | 0 ≤ y < 2n, (y ⊕ t) & m = 0} is 0 for some n-bit pattern m that has vm = k. By combining these constraints we can compute the BDD for f when n = 4 and k = 3; it has 880 nodes, and 43,146 solutions.

Similarly we have the following results, analogous to those in Section 7.1.1:

n=0n=1n=2n=3n=4n=5n=61CNF241028822447302CNF24161664,170224,71624,445,3683CNF241625643,146120,510,1324,977,694,100,656

And if we consider equivalence under complementation and permutation, the counts are:

1CNF23456782CNF23614451961,3603CNF2362225337,098109,873,815

192. (a) S(p)=Σm=0Npm(1−p)N−mQm. (b) We have ∫0N (t/N)m(1−t/N)N−mdt=NB(m+1,N−m+1)=NN+1/(Nm), by exercises 1.2.6–40 and 41; hence S¯k,n=NN+1Σm=0N⁢ qm=NN+1Sk,n. [See B. Bollobás, Random Graphs (1985), Theorem II.4.]

194. A similar question, about proofs of unsatisfiability when α > lim supn→∞ S3,n/n, is also wide open.

195. E X = 2n Pr(0 … 0 satisfies all) = 2n(1 − 2−k)m = exp(n ln 2 + m ln(1 − 2−k)) < 2 exp(−2−k−1n ln 2). Thus Sk(⌊(2k ln 2)n⌋,n) = Pr(X > 0) ≤ exp(−Ω(n)). [Discrete Applied Math. 5 (1983), 77–87. Conversely, in J. Amer. Math. Soc. 17 (2004), 947–973, D. Achlioptas and Y. Peres use the second moment principle to show that (2k ln 2 − O(k))n random kSAT clauses are almost always satisfiable by vectors x with vx ≈ n/2. Careful study of “covering assignments” (see exercise 364) leads to the sharp bounds

2k⁢ ln 2−1+ln22O(2−k3)≤lim inf n→∞αk(n)≤lim sup αk(n)≤2kln 2−1+ln 22+O(2−k3);
see A. Coja-Oghlan and K. Panagiotou, Advances in Math. 288 (2016), 985–1068.]

196. The probability is ((n−t)k¯/nk¯)αn+O(1)=e−ktα(1+O(1/n)) that αn + O(1) random kSAT clauses omit t given letters. Let p = 1 − (1 − e−kα)k. By inclusion and exclusion, the first clause will be easy with probability p(1 + O(1/n)), and the first two will both be easy with probability p^2(1 + O(1/n)). Thus if X=Σj=1m [clause j is easy], we have E X = pm+O(1) and E X^2 = p^2m^2 + O(m). Hence, by Chebyshev’s inequality, Pr(|X−pm)|≥rm=O(1/r2).

197. By Stirling’s approximation, ln q(a, b, A, B, n) = nf(a, b, A, B) + g(a, b, A, B) − 12 ln 2πn − (δan − δ(a+b)n) − (δbn − δ(b+B)n) − (δAn − δ(a+A)n) − (δBn − δ(A+B)n) − δ(a+b+A+B)n, where δn is positive and decreasing. And we must have f(a, b, A, B) ≤ 0, since q(a, b, A, B, n) ≤ 1. The O estimate is uniform when 0 < δ ≤ a, b, A, B ≤ M.

198. Consider one of the NM possible sequences of M 3SAT clauses, where N=8(n3) and M = 5n. By exercise 196 it contains g = 5(1 − (1 − e−15)3)n + O(n3/4) easy clauses, except with probability O(n−1/2). Those clauses, though rare, don’t affect the satisfiability; and all (Mg) of the ways to insert them among the r = M − g others are equally likely, so they tend to dampen the transition.

Let l ≤ r be maximum so that the first l noneasy clauses are satisfiable, and let p(l, r, g, m) be the probability that, when drawing m balls from an urn that contains g green balls and r red balls, at most l balls are red. Then S_3(m, n) = p(l, r, g, m)/NM and S_3(m′,n) = Σp(l, r, g, m′)/NM, summed over all NM sequences.

To complete the proof we shall show that
p(l,r,g,m+1)=p(l,r,g,m)−O(n−1/2)when  3.5n<m<4.5n;
hence S_3(m + 1,n) = S_3(m, n) − O(n−1/2), S_3(m, n) − S_3(m′,n) = O((m′ − m)n−1/2). Notice that p(l, r, g, m) = p(l, r, g, m+1) when m < l or m > l +g; thus we may assume that l lies between 3.4n and 4.6n. Furthermore the difference
dm=p(l,r,g,m)−p(l,r,g,m+1)=(m1) (r+g−m−1r−l−1)(r+gr)=(ml) (r+g−mr−1)(r+gr)r−lr+g−m
has a decreasing ratio dm/dm−1 = (m/(m − l))((l + g + 1 − m)/(r + g − m)) when m increases from l to l + g. So max dm occurs at m ≈ l(r + g)/r, where this ratio is ≈ 1. Now exercise 197 applies with a = l/n, b = ρg/n, A = (r−l)/n, B = (1−ρ)g/n, ρ = l/r.

[D. B. Wilson, in Random Structures & Algorithms 21 (2002), 182–195, showed that similar methods apply to many other threshold phenomena.]

199. (a) Given the required letters {a_1,...,at}, there are m ways to place the leftmost a_1, then m – 1 ways to place the leftmost a_2, and so on; then there are at most N ways to fill in each of the remaining m – t slots.

(b) By inclusion and exclusion: There are (N – k)m words that omit k of the letters.

(c) N−m∑k(tk)(−1)k∑j(mj)Nm−j(−k)j=∑j(mj)(−1)j+tN−jAj, where Aj=∑k(tk)(−1)t−kkj={jt}t! by Eq. 1.2.6–(53).

200. (a) The unsatisfiable digraph must contain a strong component with a path
l¯t→l1→…→lt→lt+1→…→ll=l¯t,
where l_1, ..., lt are strictly distinct. This path yields an s-snare (C; t, u) if we set s to the smallest index such that |ls+1| = |lu| for some u with 1 ≤ u < s.

(b) No: (x∨y)∧(y¯∨x)∧(x¯∨y) and (x∨y)∧(y¯∨x)∧(x¯∨y¯) are both satisfiable.

(c) Apply exercise 199(a) with t = s +1, N = 2n(n – 1); note that ms+1¯≤ms+1.

201. (a) Set (li,li+1) ← (x_1,x_2) or (x¯2,x¯1), where 0 ≤ i < 2t (thus 4t ways).

(b) Set (li,li+1,li+2) ← (x_1,x_2,x_3) or (x¯3,x¯2,x¯1), where 0 ≤ i < 2t; also (l¯,lt,lt+1) or (lt−1,lt,l¯2t−1)←(x1,x2,x3) or (x¯3,x¯2,x¯1) (total 4t + 4 ways, if t>2).

(c) (l_1,lt–1,lt) or (l¯2t−1,l¯t+1,l¯t)←(x1,xt−1,xt) or (x¯t−1,x¯t,xt) (4 ways).

(d) li or l¯2t−i←xi or x¯t−i, for 1 ≤ i ≤ t (4 ways, if you understand this notation).

(e) By part (a), it is 2t × 4t = 8t^2.

(f) Parts (b) and (c) combine to give N(3, 2) = (2t + 2) × (4t + 4) + 2 × 4 = 8(t^2 +2t + 2) when t > 2. From part (d), N(t, t) = 8. Also N(2t – 1, 2t) = 8; this is the number of snakes that specify the same 2t clauses. (Incidentally, when t = 5 the generating function ∑q,rN(q, r) wqzr is 1 + 200w^2z^1 + (296w^3 + 7688w^4)z^2 + (440w^4 + 12800w^5 + 55488w^6)z^3 + (640w^5 + 12592w^6 + 66560w^7 + 31104w^8)z^4 +(8w^5 + 736w^6 + 8960w^7 + 32064w^8 + 6528w^9)z^5 + (32w^6 + 704w^7 + 4904w^8 + 4512w^9)z^6 + (48w^7 + 704w^8 + 1232w^9)z^7 + (64w^8 + 376w^9)z^8 +80w^9z^9 +8w^9z^10.)

(g) The other l’s can be set in at most 22t−1−q(n−q)2t−1−q¯=R/(2qnq¯) ways.

(h) We may assume that r < 2t. The r chosen clauses divide into connected components, which are either paths or a “central” component that contains either (x¯0∨x1) and (x¯t−1∨xt) or (x¯t∨xt+1) and (x¯2t−1∨x0). Thus q equals r plus the number of components, minus 1 if the central component includes a cycle. If the central component is present, we must set lt ← xt or x¯t, and there are at most 8 ways to complete the mapping of that component. And N(r, r) = 16(r +1–t) for t < r < 2t.

Cases with k > 0 paths can be chosen in at most (2t+22k) ways, because we choose the starting and ending points, and they can be mapped in at most 2kk!(2t+22k) ways; so they contribute ∑k>0 O(t4kk/(k!3nk)) = O(t^4/n) to (2n)rpr. The noncyclic central components, which can be chosen in Θ(t^4) ways, also contribute O(t^4/n).

202. (a) m(m−1)...(m−r+1)/mr≥1−(r2)/m; (2n(n–1)–r)m–r/(2n(n–1))m–r ≥ 1 – (m – r)r/(2n(n – 1)) when r ≤ m < 2n(n – 1); and both factors are ≤ 1.

(b) The term of (95) for r = 0 is 1 plus a negligible error. The contribution of O(t^4/n) for r > 0 is O(n4/5+1/6–1), because ∑r≥0 (1 + n–1/6)–r = n1/6 + 1. And the contributions of (96) to (95) for r ≥ t are exponentially small, because in that range we have (1 + n–1/6)–t = exp(–t ln(1 + n–1/6)) = exp(–Ω(n1/30)). Finally, then, by the second moment principle MPR–(22), S_2(⌊n + n5/6⌋,n) ≤ 1 – Pr(X > 0) ≤ 1 – (E X)2/(E X^2) = 1 – 1/((E X^2)/(E X)2) = 1 – 1/(1+ O(n–1/30)) = O(n–1/30).

203. (a) E X = dn E X(1,..., 1), by symmetry; and E X(1,..., 1) = (1 – p)m, because each set of q clauses is falsified with probability p. So E X = exp((r ln(1 – p)+1)n ln d) is exponentially small when r ln(1 – p)+1 < 0; and we know that Pr(X > 0) ≤ E X.

(b) Let θs=(s2)/(n2)=s(s−1)n(n−1), and consider a random constraint set, given that X(1,..., 1) = 1. With probability θs, both u and v have color 1 and the constraint is known to be satisfied. But with probability 1–θs, it holds with probability (d2−2q)/(d2−1q). Thus ps = (θs +(1 – θs)(d^2 – pd^2 – 1)/(d^2 – 1))m.

(c) We have Pr(X > 0) ≥ dn(1 – p)m/ E(X | X(1,..., 1) = 1), from the inequality and symmetry; and the denominator is Σs=0n(ns)(d−0)n−sps. We can replace ps by the simpler value ps′=(1−p+ps2/n2)m, because ps<(θs+(1−θs)(1−p))m=(1−p+θsp)m<ps′. And we can divide the simplified sum by dn(1 – p)m.

(d) We have ∑s=03n/dts=eO(m/d2)      ∑s=03n/d(ns)  (1d)s  (1−1d)n−s, because s^2/n^2 = O(1/d^2) when s ≤ 3n/d. This sum is ≥ 1 – (e^2/27)n/d by exercise 1.2.10–22; and the crucial assumption that α>12 makes m/d^2 → 0.

(e) Transition between increase and decrease occurs when xs ≈ 1; and we have
xs=n−ss+1 1d−1(1+(2s+1)p(1−p)n2+ps2)m≈exp(ln1−σσ+(2prσ1−p+pσ2−1)  ln d)

when s = σn. Let f(σ) = 2prσ/(1 – p + pσ^2) – 1, and notice that f′(σ) > 0 for 0 ≤ σ < 1 because p≤12. Furthermore our choice of r makes f(12)<0<f(1). Setting g(σ)=f(σ)/lnσ1−σ, we seek values of σ with g(σ) = 1/ ln d. There are three such roots, because g(1/N) ≈ –f(0)/ ln N ≥ 1/ ln N; g(12±1/N)≈∓f(12)N/4; and g(1 – 1/N) ≈ f(1)/ ln N.

(f) At the second peak, where s = n – n/df(1), we have (see exercise 1.2.6–67)

ts<(nedn−s)n−s (1d)n  (1+p1−p)m=exp((−ε+O(1/df(1)))n  ln d),
which is exponentially small. And when s = 3n/d, ts<(nesd)s eO(m/d2)=O((e/3)3n/d) is also exponentially small. Consequently Σs=3n/dn⁢  ts is exponentially small.

[This derivation holds also when the random constraints are k-ary instead of binary, with q = pdk and α> 1/k. See J. Artificial Intelligence Res. 12 (2000), 93–103.]

204. (a) If the original literals ±xj that involve variable xj correspond to σ_1Xi(1), ... , σpXi(p), with signs σh, add the clauses (–σhXi(h) ∨σh+ Xi(h+)) for 1 ≤ h ≤ p to enforce consistency, where h+ = 1+(h mod p). (This transformation, due to C. A. Tovey, works even in degenerate cases. For example, if m = 1 and if the given clause is (x1∨x1∨x¯2), the transformed clauses are (X_1 ∨ X_2 ∨ X_3), (X¯1 ∨X2), (X¯2 ∨X1), (X3 ∨X¯3).)

(b) (Solution by E. Wynn.) The following 44 clauses in 35 variables are satisfiable if and only if each variable is false: ai ∨b¯i∨ci, a¯i ∨b¯i∨ci, b¯i ∨c¯i∨d¯i, c¯i ∨di∨ei, di ∨e¯i∨fi, e¯i∨f¯i∨g¯i, f¯i ∨gi∨hi, f¯i ∨gi∨h¯i, for i ∈ {1, 2}; b1∨b2∨A¯, A∨B∨C¯, A∨B¯∨D, A∨D¯∨E, B¯∨D¯∨E¯, B∨C∨F¯1, C∨E¯∨G1, C∨F1∨G¯1; Fj∨Gj∨F¯j+1, Fj∨Gj∨G¯j+1, for 1 ≤ j ≤ 6; D ∨ E ∨ ā_1, F_2 ∨ G_2 ∨ ā_2, F3∨G3∨h¯1, F4∨G4∨h¯2, a1∨h1∨d¯1, a2∨h2∨d¯2, F5∨G5∨g¯1, F6∨G6∨g¯2.

(c) Add the clauses of (b), and the clauses Fj∨Gj∨F¯j+1, Fj∨Gj∨G¯j+1 for 7 ≤ j ≤ ⌈3m/2⌉ + 3, to the 4m clauses of (a). We can stick the literals {F_7,G_7,... }, which are always false, into the 2-clauses without using any variable five times, obtaining at most (7m+39) 3-clauses in N ≈ 7m + 30 variables.

205. (a) After F_0 = {ϵ}, F_1 = F_0 ⊔ F_0, F_2 = F_0 ⊔ F_1, F_3 = F_0 ⊔ F_2, F4=F3⊔F3′, F5=F4⊔F4″, always putting the new variable into the four shortest possible clauses, we get F_5 = {345, 23¯4, 12¯3¯, 1¯2¯3¯, 3′4¯5, 2′3¯′4¯, 1′2¯′3¯′, 1¯′2¯′3¯′, 3"4"5¯, 2"3¯"4", 1"2¯"3¯", 1¯"2¯"3¯", 3′"4¯"5¯, 2′"3¯′"4¯′′, 1′"2¯′"3¯′′′, 1¯′"2¯′"3¯′′′}.

(b) Let F_0 = {ε}, F_1 = F_0 ⊔ F_0, F_2 = F_0 ⊔ F_1, F_3 = F_0 ⊔ F_2, F_4 = F_0 ⊔ F_3, F_5 = F_1 ⊔ F_4, F_6 = F_0 ⊔ F_5, F_7 = F_0 ⊔ F_6, F8=F4⊔F7′, F_9 = F_0 ⊔ F_8, F10=F7⊔F9′, F11=F7⊔F10′, F_12 = F_0 ⊔ F_11, F13=F9⊔F12″, F14=F10⊔F12(3), F15=F12⊔F14(4), F16=F13⊔F14(6), F17=F14⊔F15(7), F18=F16⊔F17(13). (Here ‘x(3)’ stands for ‘x′"’, etc.) Then F_18 consists of 257 unsatisfiable 4-clauses in 234 variables.

(Is there a shorter solution? This problem was first solved by J. Stříbrná in her M.S. thesis (Prague: Charles University, 1994), with 449 clauses. The ⊔ method was introduced by S. Hoory and S. Szeider, Theoretical Computer Science 337 (2005), 347– 359, who presented an unsatisfiable 5SAT problem that uses each variable at most 7 times. It’s not known whether 7 can be decreased to 6 when every clause has size 5.)

206. Suppose F and F′ are minimally unsatisfiable, and delete a clause of F ⊔ F′ that arose from F′; then we can satisfy F ⊔ F′ with x true.

Conversely, if F ⊔ F′ is minimally unsatisfiable, F and F′ can’t both be satisfiable. Suppose F is unsatisfiable but F′ is satisfied by L′. Removing a clause of F ⊔ F′ that arose from F′ is satisfiable only with x true; but then we can use L′ to satisfy F ⊔ F′. Hence F and F′ are both unsatisfiable. Finally, if F \ C is unsatisfiable, so is (F⊔F′)⁢ \ (C|x¯), because any solution would satisfy either F \ C or F′.

207. The five clauses of C(x,y,z;a,b,c)={xa¯b,yb¯c,zc¯a,abc,a¯b¯c¯} resolve to the single clause xyz. Thus C(x,y,y;1,2,3)∪C(x,y¯,y¯;4,5,6) ∪ C(x¯,z,z;7,8,9)∪C(x¯,z¯,z¯;a,b,c) is a solution. [K. Iwama and K. Takaki, DIMACS 35 (1997), 315–333, noted that the 16 clauses {x¯y¯z¯} ∪ C(x, x, x;1, 2, 3) ∪ C(y, y, y; 4, 5, 6) ∪ C(z, z, z; 7, 8, 9) involve each variable exactly four times, and proved that no set of twelve clauses does so.]

208. Make m clones of all but one of the 20 clauses in answer 207, and put the other 3m cloned literals into the 3m binary clauses of answer 204(a). This gives 23m 3-clauses in which every literal occurs twice, except that the 3m literals X¯i occur only once.

To complete the solution, we “pad” them with additional clauses that are always satisfiable. For example, we could introduce 3m more variables ui, with new clauses X¯iuiu¯i+1 for 1 ≤ i ≤ 3m and {u3j′⁢ u3j+1′⁢ u3j+2′,u¯3j′⁢ u¯3j+1′⁢ u¯3j+2′} for 1 ≤ j ≤ m (treating subscripts mod 3m), where ui′ denotes (i even? ui: ūi).

209. Since the multiset of kt literals in any t clauses contains at least t different variables, the “marriage theorem” (Theorem 7.5.1H) implies that we can choose a different variable in each clause, easily satisfying it. [Discr. Applied Math. 8 (1984), 85–89.]

210. [P. Berman, M. Karpinski, A. D. Scott, Electronic Colloquium on Computational Complexity (2003), TR22.] This answer uses the magic number ε = δ^7 ≈ 1/58, where δ is the smallest root of δ((1 – δ^7)6 +(1 – δ^7)7) = 1. We will assign random values to each variable so that Pr[all clauses are satisfied] > 0.

Let ηj = (1 – ε)j/((1 – ε)j +(1 – ε)13–j ), and observe that ηj ≤ δ(1 – ε)j for 0 ≤ j ≤ 13. If variable x occurs d+ times and x¯ occurs d– times, let x be true with probability ηd–, false with probability 1 – ηd– = η13–d– ≤ δ(1 – ε)13–d– ≤ δ(1 – ε)d+.

Let bad(C) = [clause C is falsified by the random assignment], and construct the lopsidependency graph for these events as in exercise 351. Then, if the literals of C = (l_1 ∨ … ∨ l_7) have contrary appearances in d_1, ... , d_7 other clauses, we have

Pr(bad(C)) ≤ (δ(1 – ε)d_1 ) ... (δ(1 – ε)d_7 ) = ε(1 – ε)d1+…+d_7 ≤ ε(1 – ε)degree(C), because C has at most d_1 + … + d_7 neighbors. Theorem L, with parameter θi = ε for each event bad(C), now tells us that Pr[all m clauses are satisfied] ≥ (1 – ε)m.

[See H. Gebauer, T. Szabó, and G. Tardos, JACM 63 (2016), 43:1–43:32, for asymptotic results that apply to kSAT as k → ∞.]

211. If m clauses in n variables are given, so that 3m = 4n, let N = 8n. Consider N “colors” named jk or jk¯, where 1 ≤ j ≤ n and k is one of the four clauses that contains ±xj. Let σ be a permutation on the colors, consisting of 4-cycles that involve the same variable, with the properties that (i) (jk)σ = jk′ for some k′ and (ii) (jk¯)σ=(jk)σ¯.

There are 4n vertices of KN named jk, having the respective color lists

L(jk,1)={jk,jk¯},L(jk,2)={jk,(jk)σ},L(jk,3)={jk¯,(jk)σ}.

The other 3m vertices of KN are named ak, bk, ck for each clause k. If that clause is, say, x2∨x¯5∨x6, the color lists are

L(ak,1)={2k,5k¯,6k},L(bk,1)=L(ck,1)={2k,2k¯,5k,5k¯,6k,6k¯};L(ak,2)={(2k)σ¯},L(bk,2)={(5k)σ¯},L(ck,2)={(6k)σ¯};L(ak,3)={(2k)σ2¯,(2k)σ},L(bk,3)={(5k)σ2¯,(5k)σ},L(ck,3)={(6k)σ2¯,(6k)σ}.

Then KN □ K_3 is list-colorable if and only if the clauses are satisfiable. (For example, (jk, 1) is colored jk ⇔ ((jk)σ, 1) is colored (jk)σ ⇔ (ak, 1) is not colored jk.)

212. (a) Let xijk = 1 if and only if Xij = k. [Note: Another equivalent problem is to find an exact cover with options { {Pij, Rik, Cjk} | pij = rik = cjk = 1}. This is a special case of 3D matching. Incidentally, the 3D matching problem can be formulated as the problem of finding a binary tensor (xijk) such that xijk ≤ yijk and xi∗∗ = x∗j∗ = x∗∗k = 1, given (yijk).]

(b) c_31 = c_32 = r_13 = r_14 = 0 forces x13∗ = 0 ≠ p_13 when r=c=(1100011000111001), p=(1010110001010011).

(c) Make L(I, J) = {1,...,N} for M < I ≤ N, 1 ≤ J ≤ N. It is well known (Theorem 7.5.1L) that a latin rectangle can always be extended to a latin square.

(d) Index everything by the set {1,...,N} ∪ ∪I,J {(I, J, K) | K ∈ L(I, J)}. The elements (I, J, K) where K = min L(I, J) are called headers. Set pij = 1 if and only if (i) i = j = (I, J, K) is not a header; or (ii) i = (I, J, K) is a header, and j = J or j = (I, J, K′) is not a header; or (iii) j = (I, J, K) is a header, and i = I or i = (I, J, K′) is not a header. Set rik = cik = 1 if and only if (i) 1 ≤ i, k ≤ N; or (ii) i = (I, J, K) and k = (I, J, K′), and if i is not a header then (K′ = K or K′ is the largest element < K in L(I, J)). [Reference: SICOMP 23 (1994), 170–184.]

213. The hinted probability is (1 – (1 – p)n′ (1 – q)n–n)m, where n′ = b_1 + … + bn. Thus if p ≤ q, every x has probability at least (1 – (1 – p)n)m of satisfying every clause. This is huge, unless n is small or m is large: If m is less than αn, where α is any constant less than 1/(1 – p), then when n > –1/ lg(1 – p) the probability (1 – (1 – p)n)m > exp(αn ln(1 – (1 – p)n)) > exp(–2(α(1 – p))n) > 1 – 2(α(1 – p))n is exponentially close to 1. Nobody needs a SAT solver for such an easy problem.

Even if, say, p = q = k/(2n), so that the average clause size is k, a clause is empty — hence unsatisfiable — with probability e–k + O(n–1); and indeed a clause has exactly r elements with the Poisson probability e–kkr/r!+ O(n–1) for fixed r. So the model isn’t very relevant. [See J. Franco, Information Proc. Letters 23 (1986), 103–106.]

214. (a) T (z) = zez + 2T (pz) (e(1–p)z – 1).

(b) If f(z)=Πm=1∞(1−e(p−1)/z/pm) and τ (z) = f(z)T (z)e–z, we have τ (z) = zf(z) + 2τ (pz) = zf(z) + 2pzf (pz) + 4p^2zf(p^2z) + … .

(c) See P. Jacquet, C. Knessl, and W. Szpankowski, Combinatorics, Probability, and Computing 23 (2014), 829–841. [The sequence 〈 Tn 〉 was first studied by A. T. Goldberg, Courant Computer Science Report 16 (1979), 48–49.]

215. Since any given x_1 ...xl is a partial solution in (8(n3)−(l3))m  of the (8(n3))m  possible cases, level l contains Pl=2l(1−18l3¯/n3¯)m nodes on the average. When m = 4n and n = 50, the largest levels are (P_31,P_32,...,P_36) ≈ (6.4, 6.9, 7.2, 7.2, 6.8, 6.2) × 10^6, and the mean total tree size P_0 + … + P_50 is about 85.6 million.

If l = 2tn and m = αn we have Pl = 2f(t)n, where f(t) = 2t + α lg(1 – t^3) + O(1/n) for 0 ≤ t ≤ 1/2. The maximum f(t) occurs when ln 4 = 3αt^2/(1 – t^3), at which point t=tα=β−12β4+58β7+O(β10), where β=ln 4/(3α); for example, t_4 ≈ 0.334. Now

PL+kPL=exp (−γk2n+O(kn)+O(k3n2)),γ=(ln2)6α(1+2tα3),when⁢  L=2tαn;
by trading tails, the expected total tree size is πn/γPL(1+O(1/n)).

[This question was first studied by C. A. Brown and P. W. Purdom, Jr., SICOMP 10 (1981), 583–593; K. M. Bugrara and C. A. Brown, Inf. Sciences 40 (1986), 21–37.]

216. If the search tree has q two-way branches, it has fewer than 2nq nodes; we shall find an upper bound on E q. Consider such branches after values have been assigned to the first l variables x_1, ... , xl, and also to s additional variables y_1, ... , ys because of unit-clause forcing; the branch therefore occurs on level t = l + s. The values can be assigned in 2t ways, and the y’s can be chosen in (n−1−ls) ways. For 1 ≤ i ≤ s the m given clauses must contain ji ≥ 1 clauses chosen (with replacement) from the F=(t−12) that force the value of yi from other known values. The other m – j_1 –…– js must be chosen from the R=8(n3)−sF−(t3)−2(t2)(n−t) remaining clauses that aren’t entirely false and don’t force anything further. Thus the expected number of two-way branches is at most

Plt=2t(n−l−1s)    Σj1,…,js≥1  (mj1,…,js,m−j)FjRm−jNm, j=j1+…+js,N=8(n3),
summed over 0 ≤ l ≤ t < n. Let b = F/N and c = R/N; the sum on j_1, ... , js is
m! [zm] (ebz−1)secz=Σr(sr)(−1)s−r(c+rb)m=s!cm Σq(mq){qs}(bc)q.

These values Plt are almost all quite small when m = 200 and n = 50, rising above 100 only when l ≥ 45 and t = 49; ΣPlt ≈ 4404.7.

If l = xn and t = yn, we have b≈38y2/n and c≈18(3(y−x)y2+y3+6y2(1−y)). The asymptotic value of [zαn] (eβz/n–1)δn eγz can be found by the saddle point method: Let ζ satisfy βδeζ/(eζ – 1) + γ = αβ/ζ, and let ρ^2 = α/ζ^2 – δeζ/(eζ – 1)2. Then the answer is approximately (eζ−1)δneγζn/βn/(2πρβ(ζn/β)αn+1).

[For exact formulas and lower bounds, see SICOMP 12 (1983), 717–733. The total time to find all solutions grows approximately as (2(78)α)n when α < 4.5, according to H.-M. Méjean, H. Morel, and G. Reynaud, SICOMP 24 (1995), 621–649.]

217. True, unless both l and l¯ belong to A or to B (making A or B tautological). For if L is a set of strictly distinct literals that covers both A and B, we know that neither A nor B nor L contains both l and l¯; hence L\{l,l¯} covers (A⁢ \{l,l¯})∪(B\  {l,l¯}=C ).

(This generalization of resolution is, however, useless if C ⊇ A or C ⊇ B, because a large clause is easier to cover than any of its subsets. Thus we generally assume that l ∈ A and l¯∈B, and that C isn’t tautological, as in the text.)

218. x? B: A. [Hence (x∨A)∧(x¯∨B) always implies A ∨ B.]

219. If C′ or C″ is tautological (℘), we define ℘ ⋄ C = C ⋄ ℘ = C. Otherwise, if there’s a unique literal l such that C′ has the form l ∨A′ and C″ has the form l¯∨A", we define C′ ⋄ C′ = A′ ∨A″ as in the text. If there are two or more such literals, strictly distinct, we define C′ ⋄ C″ = ℘. And if there are no such literals, we define C′ ⋄ C″ = C′ ∨ C″.

[This operation is obviously commutative but not associative. For example, we have (x¯⋄y¯)⋄(x∨y)=℘ while x¯⋄(y¯⋄(x∨y))=∊.]

220. (a) True: If C ⊆ C′ and C′ ⊆ C″ and C″ ≠ ℘ then C′ ≠ ℘; hence every literal of C appears in C′ and in C″. [The notion of subsumption goes back to a paper by Hugh McColl, Proc. London Math. Soc. 10 (1878), 16–28.]

(b) True: Otherwise we’d necessarily have (C ⋄ C′) ∨ α ∨ α′ ≠ ℘ and C ≠ ℘ and C′ ≠ ℘ and C ⋄ C′ ≠ C ∨ C′; hence there’s a literal l with C = l ∨ A, C′=l¯∨A′, and the literals of A ∨ A′ ∨ α ∨ α′ are strictly distinct. So the result is easily checked, whether or not α or α′ contains l or l¯. (Notice that we always have C ⋄ C′ ⊆ C ∨ C′.)

(c) False: x¯y⊆℘ but x⋄x¯y=y⊈x=x⋄℘. Also ∊⊆x¯ but x⋄∊=x⊈∊=x⋄x¯.

(d) Such examples are possible if C ≠ ∊: We have x, x¯⊢y (and also x, x¯⊢℘), although the only clauses obtainable from x and x¯ by resolution are x, x¯, and ∊. (On the other hand we do have F⊢ ∊ if and only if there’s a refutation chain (104) for F .)

(e) Given a resolution chain C1′,…,Cm+r′, we can construct another chain C_1, ... , Cm+r in which Ci⊆Ci′ for 1 ≤ i ≤ m + r. Indeed, if i > m and Ci′=Cj′⋄Ck′, it’s easy to see that either Cj ⋄ Cj or Ck ⋄ Ck or Cj ⋄ Ck will subsume Ci′.

(f) It suffices by (e) to prove this when α_1 = … = αm = α; and by induction we may assume that α = l is a single literal. Given a resolution chain C_1, ... , Cm+r we can construct another one Ci′,…,Cm+r′ such that Ci′=Ci∨l for 1 ≤ i ≤ m and Ci′⊆Ci∨l for m+1 ≤ i ≤ m+r, with Ci′=Ci′ or Ck′ or Cj′⋄Ck′ whenever Ci = Cj ⋄ Ck.

221. Algorithm A recognizes ‘1’ as a pure literal, but then finds a contradiction because the other two clauses are unsatisfiable. The resolution refutation uses only the other two clauses. (This is an example of an unnecessary branch. Indeed, a pure literal never appears in a refutation tree, because it can’t be canceled; see the next exercise.)

222. If A is an autarky that satisfies C, it also satisfies every clause on the path to from a source vertex labeled C, because all of the satisfied literals cannot simultaneously vanish. For the converse, see Discrete Appl. Math. 107 (2000), 99–137, Theorem 3.16.

223. (The author has convinced himself of this statement, but he has not been able to construct a formal proof.)

224. At every leaf labeled by an axiom A of F | x¯ that is not an axiom of F , change the label to A ∪ x; also include x in the labels of all this leaf’s ancestors. We obtain a resolution tree in which the leaves are labeled by axioms of F . The root is labeled x, if any labels have changed; otherwise it is still labeled ∊.

[See J. A. Robinson, Machine Intelligence 3 (1968), 77–94.]

225. Let’s say that a regular resolution tree for clause A is awkward if at least one of its nodes resolves on one of the variables in A. An awkward tree T for A can always be transformed into a regular non-awkward tree T′ for some clause A′ ⊆ A, where T′ is smaller than T . Proof: Suppose T is awkward, but none of its subtrees are. Without loss of generality we can find a sequence of subtrees T_0, ... , Tp, T1′,…,Tp′, where T_0 = T and Tj–1 for 1 ≤ j ≤ p is obtained from Tj and Tj′ by resolving on the variable xj; furthermore xp ∈ A. We can assume that the labels of Tj and Tj′ are Aj and Aj′, where Aj = xj ∪ Rj and Aj′=x¯j∪Rj′; hence Aj−1=Rj∪Rj′. Let Bp = Ap; and for j = p – 1, p – 2, ... , 1, let Bj = Bj+1 if xj ∉ Bj+1, otherwise obtain Bj by resolving Bj+1 with Aj′. It follows by induction that Bj ⊆ xp ∪ Aj–1. Thus B_1 ⊆ xp ∪ A_0 = A, and we’ve derived B_1 with a non-awkward tree smaller than T.

Now we can prove more than was asked: If T is any resolution tree that derives clause A, and if A ∪ B is any clause that contains A, there’s a non-awkward regular resolution tree Tr no larger than T that derives some clause C ⊆ A ∪ B. The proof is by induction on the size of T : Suppose A = A′ ∪ A″ is obtained at the root of T by resolving the clauses x ∪ A′ with x¯∪A" that label the subtrees T′ and T″. Find non-awkward regular trees Tr′ and Tr″ that derive C′ and C″, where C′ ⊆ x∪A′ ∪B and C"⊆x¯∪A"∪B. If x ∈ C′ and x¯∈C″, we obtain the desired Tr by resolving Tr′ and Tr″ on x. Otherwise we can either let C = C′ and Tr=Tr′, or C = C″ and Tr=Tr″. [It’s interesting to apply this construction to the highly irregular resolutions in (105).]

226. Initially α is the root, C(α) = ∊, ||α|| = N, and s = 0. If α isn’t a leaf, we have C(α) = C(α′) ⋄ C(α″) where x ∈ C(α′) and x¯∈C(a″) for some variable x. The Prover names x, and changes α ← α′ or α ← α″ if the Delayer sets x ← 0 or x ← 1, respectively. Otherwise min(||α′||, ||α″||) ≤ ||α||/2, and the Prover can keep going.

227. The proof is by induction on the number of variables, n: If F contains the empty clause, the game is over, the Delayer has scored 0, and the root is labeled 0. Otherwise the Prover names x, and the Delayer considers the smallest possible labels (m, m′) on the roots of refutations for F | x and F | x¯. If m > m′, the reply x ← 0 guarantees m points; and the reply x ← ∗ is no better, because m′ + 1 ≤ m. If m < m′, the reply x ← 1 guarantees m′; and if m = m′, the reply x ← ∗ guarantees m +1. Thus an optimum Delayer can always score at least as many points as the root label of any branch of a refutation tree constructed by the Prover. Conversely, if the Prover always names an optimal x, the Delayer can’t do better.

(This exercise was suggested by O. Kullmann. One can compute the optimum score “bottom up” by considering all 3n possible partial assignments as in answer 133.)

228. We need only assume the transitivity clauses Tijk of (100) when i < j and k < j. [Notice further that Tijk is tautological when i = j or k = j, thus useless for resolution.]

229. Using the binary-relation interpretation, these clauses say that j ⊀ j, that the transitive law “i ≺ j and j ≺ k implies i ≺ k” holds whenever i ≤ k and j < k, and that every j has a successor such that j ≺ k. The latter axiom combines with the finiteness of m to imply that there must be a cycle j_0 ≺ j_1 ≺ … ≺ jp–1 ≺ jp = j_0.

Consider the shortest such cycle, and renumber the subscripts so that jp = max{j_0,...,jp}. We cannot have p ≥ 2, because (100′) implies jp–2 ≺ jp, yielding a shorter cycle. Hence p = 1; but that contradicts (99).

230. Call the axioms Ij, Tijk, and Mjm as in the text. If Ij_0 is omitted, let xij = [j = j_0] for all i and j. If Ti_0j_0k_0 is omitted, let xij = [j ∈ A] for all i∉ A = {i_0, j_0, k_0}; also let xi_0j = [j = j_0], xj_0j = [j = k_0 ], and (if i_0 ≠ k_0) xk_0j = [j = i_0]. Finally, if Mj_0m is omitted, let xij = [pi <pj ], where p_1 ...pm = 1 ... (j_0–1)(j0+1) ...mj_0. (The same construction shows that the clauses of answer 228 are minimally unsatisfiable.)

231. Since G_11 = M_1m, we can assume that j > 1. Then G(j–1)j = G(j–1)(j–1) ⋄ Ij–1. And if 1 ≤ i < j – 1 we have Gij = (… ((G(j–1)j ⋄ Aijj) ⋄ Aij(j+1)) ⋄ …) ⋄ Aijm, where Aijk=Gi(j−1)⋄Ti(j−1)k=Gij∨x¯(j−1)k. These clauses make it possible to derive Bij=(…((Gij⋄Tiji)⋄Tji(j+1))⋄…)⋄Tjim=Gjj∨x¯ji for 1 ≤ i < j, from which we obtain Gjj = (… ((Mjm ⋄ B_1j) ⋄ B_2j) ⋄ …) ⋄ B(j–1)j. Finally Gmm ⋄ Im = ∊.

232. It suffices to exhibit a backtrack tree of depth 6 lg q + O(1). By branching on at most 6 variables we can find the color-triplet α_1 in answer 176(c).

Suppose we know that αj = α and αj+p = α′, where α′ cannot be obtained from α in p steps; this is initially true with j = 1, α = α′ = α_1, and p = q. If p = 1, a few more branches will find a contradiction. Otherwise at most 6 branches will determine αl, where l = j + ⌊p/2⌋; and either αl will be unreachable from α in ⌊p/2⌋ steps, or α′ will be unreachable from αl in ⌈p/2⌉ steps, or both. Recurse.

233. C_9 = C_6 ⋄ C_8, C_10 = C_1 ⋄ C_9, C_11 = C_3 ⋄ C_10, C_12 = C_7 ⋄ C_10, C_13 = C_4 ⋄ C_11, C_14 = C_2 ⋄ C_12, C_15 = C_13 ⋄ C_14, C_16 = C_5 ⋄ C_15, C_17 = C_6 ⋄ C_15, C_18 = C_8 ⋄ C_15, C_19 = C_12 ⋄ C_17, C_20 = C_11 ⋄ C_18, C_21 = C_16 ⋄ C_19, C_22 = C_20 ⋄ C_21.

234. Reply xjk ← ∗ to any query that doesn’t allow the Prover to violate (107). Then the Prover can violate (106) only after every hole has been queried.

235. Let C(k,A)=(∨j=0k ∨a∈A xja), so that C(0, {1,...,m}) = (x_01 ∨…∨ x_0m) and C(m, ∅) = ∊. The chain consists of m stages for k = 1, ... , m, where stage k begins by deriving the clauses x¯ka∨C(k−1,A) from the clauses of stage k – 1, for all (m – k)-element subsets A of {1,...,m}\ a; every such clause requires k resolutions with (107). Stage k concludes by deriving C(k, A) for all (m – k)-element subsets A of {1,...,m}, each using one resolution from (106) and k – 1 resolutions from the beginning of the stage. (See (103).) Thus stage k involves a total of (mm−k)(k2+k) resolutions.

For example, the resolutions when m = 3 successively yield 11¯ 02 03, 12¯ 01 03, 13¯ 01 02; 01 02 11 12, 01 03 11 13, 02 03 12 13 (stage 1); 21¯ 02 11 12, 21¯ 02 12, 21¯ 03 11 13, 21¯ 03 13, 22¯ 01 12 11, 22¯ 01 11, 22¯ 03 12 13, 22¯ 03 13, 23¯ 01 13 11, 23¯ 01 11, 23¯ 02 13 12, 23¯ 02 12; 01 11 21 22, 01 11 21, 02 12 22 23, 02 12 22, 03 13 23 22, 03 13 23 (stage 2); and 31¯ 11 21, 31¯ 21, 31¯, 32¯ 12 22, 32¯ 22, 32, 33¯ 13 23, 33¯ 23, 33¯; 32 33, 33, ∊ (stage 3).

[Stephen A. Cook constructed such chains in 1972 (unpublished).]

236. The symmetry of the axioms should allow exhaustive verification by computer for m = 2, possibly also for m = 3. The construction certainly seems hard to beat. Cook conjectured in 1972 that any minimum-length resolution proof would include, for every subset S of {1,...,m}, at least one clause C such that ∪±xjk ∈C {k} = S.

237. The idea is to define yjk = xjk ∨ (xjm ∧ xmk) for 0 ≤ j < m and 1 ≤ k < m, thus reducing from m + 1 pigeons to m. First we append 6m(m – 1) new clauses

(xjm∨zjk)∧(xmk∨zjk)∧(x¯jm∨x¯mk∨z¯jk)∧(x¯jk∨yjk)∧(yjk∨zjk)∧(xjk∨y¯jk∨z¯jk),
involving 2m(m – 1) new variables yjk and zjk. Call these clauses Ajk, ... , Fjk.

Now if Pj stands for (106) and Hijk for (107), we want to use resolution to derive Pj′=(yj1∨…∨yj(m−1)) and Hijk′=(y¯ik∨y¯jk). First, Pj can be resolved with Dj1, ... , Dj(m–1) to get Pj′∨xjm. Next, Pm⋄Hjmm=xm1∨…∨xm(m−1)∨x¯jm can be resolved with Gjk=Cjk⋄Ejk=x¯jm∨x¯mk∨yjk for 1 ≤ k < m to get Pj′∨x¯jm. One more step yields Pj′. (The intuitive “meaning” guides these maneuvers.)

From Bjk⋄Fjk=xjk∨xmk∨y¯jk, we obtain Qijk=x¯ik∨y¯jk after resolving with Hijk and Himk. Then (Qijk⋄Fik)⋄Aik=xim∨y¯ik∨y¯jk=Rijk, say. Finally, (Rjik⋄Hijm)⋄Rijk=Hijk′ as desired. (When forming Rjik we need Qjik with j > i.)

We’ve done 5m^3 – 6m^2 +3m resolutions to reduce m +1 to m. Repeating until m = 0, with fresh y and z variables each time, yields ∊ after about 54m4 steps.

[See Stephen A. Cook, SIGACT News 8, 4 (October 1976), 28–32.]

238. The function (1 – cx)–x = exp(cx^2 + c^2x^3/2 + …) is increasing and > ecx^2. Setting c=12n, W=2n  ln r, and b = ⌈W⌉ makes f ≤ r < ρ–b. Also W ≥ w(α_0) when n ≥ w(α_0)2 and r ≥ 2; hence w(α0⊢∊)≤W+b≤8n  ln   r+1 as desired. The ‘–2’ in the lemma handles the trivial cases that arise when r < 2.

(It is important to realize that we don’t change n or W in the induction proof. Incidentally, the exact minimum of W + b, subject to r = (1 – W/(2n))–b, occurs when

W=2n(1−e−2T(z))=4nz+2nz33+…,b=ln r2T(z)=(lnr)(12z−12−z4−…),
where z^2 = (ln r)/(8n) and T (z) is the tree function. Thus it appears likely that the proof of Lemma B supports the stronger result w(α0⊢∊)<8n  ln   r−12 ln r+1.)

239. Let α_0 consist of all 2n nontautological clauses of length n. The shortest refutation is the complete binary tree with these leaves, because every nontautological clause must appear. Algorithm A shows that 2n – 1 resolutions suffice to refute any clauses in n variables; hence || α_0 ⊢ ∊ || = 2n – 1, and this is the worst case.

240. If A′ has t elements and ∂A′ has fewer than t, the sequence of 5t integers fij for its neighbors must include at least 2t repeats of values seen earlier. (In fact there are at least 2t + 1 repeats, because 2t would leave at least t in the boundary; but the calculations are simpler with 2t, and we need only a rather crude bound.)

The probability pt that some such A′ exists is therefore less than (m+1t)(5t2t)(3tm)2t, because there are (m+1t) ways to select A′, (5t2t) to select the repeating slots, and at most (3t)2t out of m2t ways to fill those slots. Also (m+tt)=(mt)+(mt−1)<2(mt) when t≤12m.

By exercise 1.2.6–67 we have pt≤2(met)t(5te2t)2t(3tm)2t=2(ct/m)t,where c = 225e^3/4 ≈ 1130. Also p_0 = p_1 = 0. Thus the sum of pt for t ≤ m/3000 is less than 2 Σt=2∞(c/3000)t≈.455; and the probability of strong expansion exceeds .544.

241. If 0 < |A′| ≤ m/3000, we can put one of its elements into a hole bk ∈ ∂A′. Then we can place the other elements in the same way, since bk isn’t their neighbor.

242. The proof of Theorem B remains valid when these new axioms are added.

243. (a) The probability that F′ has t elements and V (F′) has fewer than t is at most (αnt)(nt)(tn)3t≤(αe2tn)t. The sum of this quantity for 1 ≤ t ≤ lg n is O(n–1), and so is the sum for lg n ≤ t ≤ n/(2αe^2).

(b) If the condition in (a) holds, there’s a matching from F′ into V (F′), by Theorem 7.5.1H; hence we can satisfy F′ by assigning to its variables, one by one. If F is unsatisfiable we’ll therefore need to invoke more than n/(2αe^2) of its axioms.

(c) The probability pt that F′ has t elements and 2|V(F′)|−3|F′|<12|F′| is at most (αnt)(nλt)(λtn)3t≤(αe1+λλ3−λ(t/n)1/4)t, where λ=74. We have (e1+λλ3–λ)4 < 10^6; so pt < ct when t ≤ n′, where c < 1, and Σt=n′/2n′pt is exponentially small.

(d) Since n′ < n/(2αe^2), every refutation a.s. contains a clause C with n′/2 ≤ μ(C) < n′. The minimal axioms F′ on which C depends have |F′| = μ(C). Let k be the number of “boundary” variables that occur in just one axiom of F′. If v is such a variable, we can falsify C and the axiom containing v, while the other axioms of F′ are true; hence V must contain v or v¯. We have |V(F′)|=k+|nonboundary|≤k+12(3|F′|−k), because each nonboundary variable occurs at least twice. Therefore k ≥ 2|V (F′)|– 3|F′| ≥ n′/4, q.s. (Notice the similarities to the proof of Theorem B.)

244. We have [A ∪ B]0 = [A]0[B]0 ∪ [A]1[B]1 and [A ∪ B]1 = [A]0[B]1 ∪ [A]1[B]0, where concatenation of sets has the obvious meaning. These relations hold also when A = ∅ or B = ∅, because [∅]0 = {∊} and [∅]1 = ∅.

245. (a) When conditioning on ēuv, simply delete the edge u ––– v from G. When conditioning on euv, also complement l(u) and l(v). The graph might become disconnected; in that case, there will be exactly two components, one even and one odd, with respect to the sums of their labels. The axioms for the even component are satisfiable and may be discarded.

For example, α(G) |{b, ē} corresponds to while α(G) |{b, e} corresponds to . We toss out the left component in the first case, the right one in the other.

(b) If C ∈ α(v) we may take V′ = {v}. And we have μ(∊) = |V |, because the axioms ∪v∈V\u α(v) are satisfiable for all u ∈ V.

(c) If u ∈ V′ and v ∉ V′, there’s an assignment that falsifies C and some axiom of α(u) while satisfying all α(w) for w ∈ V′ \ u, because |V′| is minimum. Setting euv ← ēuv will satisfy α(u) without affecting the axioms α(w) (which don’t contain euv).

(d) By (b), every refutation of α(G) must contain a clause C with 13m≤μ(C)<23m. The corresponding V′ has |V′|/(|V′|+|∂V′|)<(23+8)/9, hence |∂V′|>126|V′|.

[Property (i) is interesting but irrelevant for this proof. Notice that α(G) has exactly 83n≈2.67n 3SAT clauses in n = 3m/2 variables when G is cubic; every literal occurs four times. G. Tseytin proved lower bounds for refutations of α(G) by regular resolution in 1966, before graphs with property (iii) were known; A. Urquhart obtained them for general resolution in JACM 34 (1987), 209–219, and the simplified argument given here is due to Ben-Sasson and Wigderson. The fact that α(G) requires exponentially long refutation chains, although the same axioms can be refuted easily by working with linear equations mod 2, amounts to a proof that backtracking is a poor way to deal with linear equations! Suitable Ramanujan graphs raman (2,q, 3, 0) can be found by the algorithms of the Stanford GraphBase for infinitely many prime numbers q. We can also obtain the same lower bounds with the multigraphs raman (2,q, 1, 0) and raman (2,q, 2, 0). Section 7.4.3 will explore expander graphs in detail.]

246. Let’s write [a_1 ...ak]ℓ for what exercise 244 calls [{a_1,...,ak}]ℓ. With new variables x, y, z we can introduce {xa,xb¯,x¯a¯b,ya¯,yb,y¯ab¯,zx,zy,z¯x¯y¯} and resolve those clauses to [zab]1, which means z = a ⊕ b. So we can assume that ‘z ← a ⊕ b’ is a legal primitive operation of “extended resolution hardware,” when z is a new variable. Furthermore we can compute a_1 ⊕…⊕ ak in O(k) steps, using z_0 ← 0 (which is the clause [z_0]1, namely z¯0) and zk ← zk–1 ⊕ ak when k ≥ 1.

Let the edge variables E(v)be a_1, ... , ad, where d is the degree of v. We compute sv ← a_1 ⊕… ⊕ ad by setting sv,0 ← 0, sv,k ← sv,k–1 ⊕ ak, and sv ← sv,d. We can resolve sv with the axioms α(v) in O(2d) steps, to get the singleton clause [sv]l(v)⊕1, meaning sv = l(v). Summing over v, these operations therefore take O(N) steps.

On the other hand, we can also compute zn ← ⊕v sv and get zero (namely ‘z¯n’). Doing this cleverly, by omnisciently knowing G, we can in fact compute it in O(mn) steps: Start with any vertex v and set z_1 ← sv (more precisely, set z_1,k ← sv,k for 0 ≤ k ≤ d). Given zj for 1 ≤ j < n, with all its subvariables zj,k, we then compute zj+1 ← zj ⊕su, where u is the unused vertex with su,1 = zj,1. We can arrange the edges into an order so that if zj has p edge variables in common with su, then zj,k = su,k for 1 ≤ k ≤ p. Suppose the other variables of zj and su are respectively a_1, ... , aq and b_1, ... , br; we want to merge them into the sequence c_1, ... , cq+r that will be needed later when zj+1 is used. So we set zj+1,0 ← 0, zj+1,k ← zj+1,k–1 ⊕ ck, zj+1 ← zj+1,q+r.

From the clauses constructed in the previous paragraph, resolution can deduce [zj,ksu,k]1 for 1 ≤ k ≤ p, and hence [zj+1,0zj,psu,p]1 (namely that zj+1,0 = zj,p ⊕ su,p). Furthermore, if ck = ai, and if we know that zj+1,k–1 = zj,s ⊕ su,t where s = p + i – 1and t = p + k – i, resolution can deduce that zj+1,k = zj,s+1 ⊕ su,t; a similar formula applies when ck = bi. Thus resolution yields zj+1 ← zj ⊕ su as desired. Ultimately we deduce both zn and z¯n from the singleton clauses sv = l(v).

247. Eliminating x_2 from {12, 1¯2, 1¯2¯} gives {1¯}; eliminating x_1 then gives ∅. So those five clauses are satisfiable.

248. We have F(x1,…,xn)=(xn∨A1′)∧…∧(xn∨Ap′)∧(x¯n∨A1″) ∧ … ∧ (x¯n∨Aq″)∧A1′″∧…∧Ar′″=(xn∨G′)∧(x¯n∨G″)∧G′″, where G′=A1′∧…∧Ap′, G″=A1″∧…∧Aq″, and G′″=A1′″∧…∧Ar′″ depend only on {x_1,...,xn–1}. Hence F′ = (G′ ∨ G″) ∧ G′" ; and the clauses of G′∨G″=∧i=1p  ∧j=1q (Ai′∨Aj″) are the resolvents eliminating xn.

249. After learning C7=2¯3¯ as in the text, we set d ← 2, l2←2¯, Cj=23¯, learn C8=3¯, and set d ← 1, l1←3¯. Then l2←4¯ (say); and l3←1¯, l4←2¯. Now Ci = 1234 has been falsified; after l_4 ← 2 and Cj=12¯ we learn C_9 = 134, set l_3 ← 1, and learn C10=134⋄1¯3=34. Finally l_2 ← 4, we learn C_11 = 3; l_1 ← 3, and we learn C_12 = ∊.

250. l_1 ← 1, l_2 ← 3, l3←2¯, l_4 ← 4; learn 1¯23¯; l_3 ← 2, l_4 ← 4; learn 1¯2¯3¯ and 1¯3¯; l2←3¯, l3←2¯, l_4 ← 4; learn 1¯23; l_3 ← 2, l_4 ← 4; learn 1¯2¯3, 1¯3, 1¯; l1←1¯, l_2 ← 3, l3←4¯, l_4 ← 2; learn 13¯4; l_3 ← 4, l4←2¯, l_4 ← 2.

251. Algorithm I has the property that l¯i1, ..., l¯ik−1, lik are on the stack whenever the new clause li_1 ∨…∨ lik has been learned, if i_1 < … < ik = d and step I4 returns to I2. These literals limit our ability to exploit the new clause; so it appears to be impossible to solve this problem without doing more resolutions than Stålmarck did.

However, we can proceed as follows. Let Mimk" be the clause xm_1 ∨...∨ xm(k–1) ∨ xik ∨… ∨ xi(m–1) ∨ x¯im, for 1 ≤ i, k < m. Using ij to stand for xij, the process for m = 3 begins by putting 11¯, 12¯, 13, 21¯, 22¯, 23, 31¯, 32¯, 33 on the stack. Then step I3 has Ci = I_3, step I4 has Cj = M_33; so step I5 learns I_3 ⋄ M_33 = M_32. Step I4 now changes 32¯ to 32 and chooses Cj = T_232; so I5 learns M32⋄T232=M232″. Step I4 changes 31¯ to 31 and chooses Cj = T_231; now we learn M232"⋄T231=M231". Next, we learn M231"⋄M23=M22; and after changing 22¯ to 22 we also learn M_21.

The stack now contains 11¯, 12¯, 13, 21. We add 31¯, 32¯, and proceed to learn M32⋄T132=M132", M132"⋄T131=M131", M131"⋄M13=M12. The stack now contains 11¯, 12, and we’ve essentially reduced m from 3 to 2.

In a similar way, O(m^2) resolutions will learn Mi(m–1) for i = m – 1, ..., 1; and they’ll leave x¯11, ..., x¯1(m−2), x1(m–1) on the stack so that the process can continue.

252. No; large numbers of clauses such as x¯12∨x¯23∨…∨x¯89∨x19 are generated by the elimination process. Although these clauses are valid, they’re not really helpful.

Exercise 373 proves, however, that the proof is completed in polynomial time if we restrict consideration to the transitivity clauses of exercise 228(!).

253. A conflict arises when we follow a chain of forced moves:

t

Lt

level

reason

0

6¯

1

Λ

1

4

1

46

2

5

2

Λ

3

3¯

2

3¯4¯5¯

4

9

2

369

5

7¯

2

5¯7¯9¯

6

1¯

2

1¯5¯9¯

7

8

2

678

8

2

2

123

9

2¯

2

2¯5¯8¯

Now 2¯5¯8¯→2¯5¯8¯⋄123=135¯8¯→135¯67→35¯679¯→35¯69¯→35¯6→4¯5¯6; so we learn 4¯5¯6 (which can be simplified to 5¯6, because 4¯ is “redundant” as explained in exercise 257).

Setting L2←5¯, with reason 4¯5¯6 or 5¯6, now forces 7, 1¯, 3, 9, 2¯, 8¯, 8, all at level 1; this conflict soon allows us to learn the unit clause 6. (Next we’ll inaugurate level 0, setting L_0 ← 6. No “reasons” need to be given at level 0.)

254. Deducing 3, 2, 4, 4¯ at level 1, it will find 2¯4¯⋄43¯=2¯3¯ and 2¯3¯⋄23¯=3¯, learning 3¯. (Or it might learn 3¯ after deducing 2¯.) Then it will deduce 3¯, 1¯, 2, 4¯ at level 0.

255. For example, {1¯2¯4¯, 2¯3¯5¯, 456, 456¯}. [Since the clause c′ that is learned by the procedure described in the text contains just one literal l from the conflict level d, the trail position for l¯ has been called a “unique implication point” (UIP). If l isn’t the decision literal for its level, we could resolve c′ with l’s reason and find another UIP; but each new resolution potentially increases the b array and limits the amount of backjumping. Therefore we stop at the first UIP.]

256. If it is false, literals 50, 26, ... , 30 are true; hence also 25¯, 23, and 29, a conflict. Consequently we can obtain ‘∗∗’ by starting with 23¯⁢  26¯…50¯ and resolving with 23 25 27, 25 27 29, and 25¯⁢  30¯…70¯. [Similarly, and more simply, one can learn (122) by resolving 11¯⁢  16¯…56¯ with 31 61 91, 41 66 91, and 56 61 66.]

257. (a) Suppose l¯′ on level d′ > 0 is redundant. Then some l″ in the reason for l′ is also on level d′; and l″ is either in c or redundant. Use induction on trail position.

(b) We can assume that the stamp value s used when resolving conflicts is a multiple of 3, and that all stamps are ≤ s. Then we can stamp literal l with S(|l|) ← s +1 if l¯ is known to be redundant, or s +2 if l¯ is known to be nonredundant and not in c. (These stamps serve as a “memo cache” to avoid repeated work.) While building c we can also stamp levels as well as literals, setting LS[d′] ← s if level d′ has exactly one of the bi, or s + 1 if it has more than one.

Then for 1 ≤ j ≤ r, b¯j is redundant if and only if LS[lev (bj)] = s + 1 and red (b¯j) is true, where lev (l) = VAL(|l|) ≫ 1 and where red (l¯) is the following recursive procedure: “If l is a decision literal, return false. Otherwise let (l ∨ ā_1 ∨…∨ āk) be l’s reason. For 1 ≤ i ≤ k with lev (ai) > 0, if S(|ai|) = s + 2 return false; if S(|ai|) < s and either LS[lev (ai)] < s or red (āi) is false, set S(|ai|) ← s + 2 and return false. But if none of these conditions hold, set S(|l|) ← s + 1 and return true.”

[See Allen Van Gelder, LNCS 5584 (2009), 141–146.]

258. That statement is true in Table 3, but false in general. Indeed, consider the sequel to Table 3: The decision L44=57¯ causes the watch list of 57 to be examined, thus forcing 15, 78, and 87 (among other literals) in some order because of the clauses 15 57 36, 78 57 36, 87 57 27. Then 96¯ will be forced by the clause 96¯  87¯…15¯; and the second literal of that clause at the time of forcing will be 15¯, regardless of trail order, if the watched literals of that clause were 96¯ and 15¯ (making it invisible to 78¯ and 87¯).

259. 1+ ρ^6 + ρ^7 < ρ + ρ^2 when .7245 < ρ < .7548. (There can in fact be any number of crossover points: Consider the polynomial (1 – ρ – ρ^2)(1 – ρ^3 – ρ^6)(1 – ρ^9 – ρ^18).)

260. First, to get a random permutation in the heap we can use a variant of Algorithm 3.4.2P: For k ← 1, 2, ... , n, let j be a random integer in [0 ..k – 1] and set HEAP[k – 1] ← HEAP[j], HEAP[j] ← k. Then set HLOC(HEAP[j]) ← j for 0 ≤ j < n.

Next, set F ← 0 and Wl ← 0 for 2 ≤ l ≤ 2n + 1 and c ← 3. Do the following for each input clause l_0l_1 ...lk–1: Terminate unsuccessfully if k = 0, or if k = 1 and 0 ≤ VAL(|l_0|) ≠ l_0 &1. If k = 1 and VAL(|l_0|) < 0, set VAL(|l_0|) ← l_0 &1, TLOC(|l_0|) ← F, F ← F + 1. If k > 1, set MEM[c + j] ← lj for 0 ≤ j < k; also MEM[c – 1] ← k, MEM[c – 2] ← Wl_0, Wl_0 ← c, MEM[c – 3] ← Wl_1, Wl_1 ← c, c ← c + k +3.

Finally, set MINL ← MAXL ← c+2 (allowing two cells for extra data in the preamble of the first learned clause). Of course we must also ensure that MEM is large enough.

261. (Throughout this answer, lj is an abbreviation for MEM[c + j].) Set q ← 0 and c←Wl¯. While c ≠ 0, do the following: Set l′ ← l_0. If l′≠l¯ (hence l1=l¯), set c′ ← l–3; otherwise set l′ ← l_1, l_0 ← l′, l1←l¯, c′ ← l–2, l–2 ← l–3, and l–3 ← c′. If VAL(|l_0|) ≥ 0 and VAL(|l_0|) + l_0 is even (that is, if l_0 is true), perform the steps
if  q≠0,set   MEM[q−3]←c,else  set  Wl¯←c; then   set   q←c.(∗)

Otherwise set j ← 2; while j < l–1 and VAL(|lj|) ≥ 0 and VAL(|lj|) + lj is odd, set j ← j+1. If now j < l–1, set l_1 ← lj, lj←l¯, l–3 ← Wl_1, Wl_1 ← c. But if j = l–1, do(∗) above; jump to C7 if VAL(|l_0|) ≥ 0; otherwise set LF ← l_0, etc. (see step C4) and c ← c′.

Finally, when c = 0, do (∗) above to terminate l¯’s new watch list.

262. To delete k = HEAP[0] in C6: Set h ← h – 1 and HLOC(k) ← –1. Stop if h = 0. Otherwise set i ← HEAP[h], α ← ACT(i), j ← 0, j′ ← 1, and do the following while j′ < h: Set α′ ← ACT(HEAP[j′]); if j′ +1 < h and ACT(HEAP[j′ +1]) > α′, set j′ ← j′+1 and α′ ← ACT(HEAP[j′]);if α ≥ α′, set j′ ← h, otherwise set HEAP[j] ← HEAP[j′], HLOC(HEAP[j′]) ← j, j ← j′, and j′ ← 2j + 1. Then set HEAP[j] ← i and HLOC(i) ← j.

In C7, set k ← |l|, α ← ACT(k), ACT(k) ← α + DEL, j ← HLOC(k), and if j > 0 perform the “siftup” operation: “Looping repeatedly, set j′ ← (j – 1) ≫ 1 and i ← HEAP[j′], exit if ACT(i) ≥ α, else set HEAP[j] ← i, HLOC(i) ← j, j ← j′, and exit if j = 0. Then set HEAP[j] ← k and HLOC(k) ← j.”

To insert k in C8, set α ← ACT(k), j ← h, h ← h +1; if j = 0 set HEAP[0] ← k and HLOC(k) ← 0; otherwise perform the siftup operation.

263. (This answer also sets the level stamps LS[d] needed in answer 257, assuming that the LS array is initially zero.) Let “bump l” mean “increase ACT(|l|) by DEL” as in answer 262. Also let blit (l) be the following subroutine: “If S(|l|) = s, do nothing. Otherwise set S(|l|) ← s, p ← lev (l). If p > 0, bump l; then if p = d, set q ← q + 1; else set r ← r +1, br←l¯, d′ ← max(d′,p), and if LS[p] ≤ s set LS[p] ← s +[LS[p] = s].”

When step C7 is entered from C4, assuming that d > 0, set d′ ← q ← r ← 0, s ← s + 3, S(|l_0|) ← s, bump l_0, and do blit (lj) for 1 ≤ j < k. Also set t ← max(TLOC(|l_0|),..., TLOC(|lk–1|)). Then, while q > 0, set l ← Lt, t ← t – 1; if S(|l|) = s then set q ← q – 1, and if Rl ≠ Λ let clause Rl be l_0l_1 ...lk–1 and do blit (lj) for 1 ≤ j < k. Finally set l′ ← Lt, and while S(|l′|) ≠ s set t ← t – 1 and l′ ← Lt.

The new clause can now be checked for redundancies as in answer 257. To install it during step C9, there’s a subtle point: We must watch a literal that was defined on level d′. Thus we set c←MAXL, MEM[c]←l¯′, k ← 0, j′ ← 1; and for 1 ≤ j ≤ r if S(|bj|) = s set k ← k+1 and do this: If j′ = 0or lev (bj) < d′, set MEM[c+k+j′]←b¯j; otherwise set MEM[c+1]←b¯j, j′ ← 0, MEM[c−2]←Wl¯′, Wl¯′←c, MEM[c−3]←Wb¯j, Wb¯j←c. Finally set MEM[c–1] ← k +1, MAXL ← c + k +6.

264. We can maintain a “history code” array, setting HF to 0, 2, 4, or 6 when LF is set, and then using Ht +(Lt & 1) as the move code that represents trail location t for 0 ≤ t < F . History codes 6, 4, and 0 are appropriate in steps C1, C4, and C6, respectively; in C9, use code 2 if l′ was a decision literal, otherwise use code 6.

[These move codes do not increase lexicographically when the trail is flushed and restarted; hence they don’t reveal progress as nicely as they do in the other algorithms.]

265. (1) A literal Lt on the trail with G ≤ t < F has become true, but the watch list of L¯t has not yet been examined. (2) If l_0 is true, so that c is satisfied, step C4 doesn’t remove c from the watch list of l_1 when l_1 becomes false. (This behavior is justified, because c won’t be examined again until l_1 has become free during the backtracking step C8.) (3) A clause that becomes a reason for l_0 remains on the watch list of its false l_1. (4) During a full run, a clause that triggers a conflict is allowed to keep both of its watched literals false.

In general, a false watched literal must be defined at the highest trail level of all literals in its clause.

266. If U < p, where U is a uniform deviate between 0 and 1, do this: Set j to a random integer with 0 ≤ j < h, and k ← HEAP[j]. If j = 0, or if VAL(k) ≥ 0, use the normal C6. Otherwise branch on k (and don’t bother to remove k from the heap).

267. As in Algorithm L, let there be a sequential table BIMP(l) for each literal l, containing all literals l′ such that l¯∨l′ is a binary clause. Furthermore, when the propagation algorithm sets LF ← l′ because l′ ∈ BIMP(l), we may set Rl ← –l, instead of using a positive clause number as the “reason.” (Notice that a binary clause therefore need not be represented explicitly in MEM, if it is represented implicitly in the BIMP tables. The author’s implementation of Algorithm C uses BIMP tables only to expedite binary clauses that appear in the original input. This has the advantage of simplicity, since the exact amount of necessary space can be allocated permanently for each table. Learned binary clauses are comparatively rare in practice; thus they can usually be handled satisfactorily with watched literals, instead of by providing the elaborate buddy-system scheme that was important in Algorithm L.)

Here, more precisely, is how the inner loop goes faster with BIMPs. We want to carry out binary propagations as soon as possible, because of their speed; hence we introduce a breadth-first exploration process analogous to (62):

Set  H←F;take account  ofl′ for  all   l′∈BIMP  (l0);while   H<F, set  l0←LH,H←H+1,and(∗∗)take  account  of   l′  for  all  l′  ∈BIMP  (l0).

Now “take account of l′” means “if l′ is true, do nothing; if l′ is false, go to C7 with conflict clause l¯∨l′; otherwise set LF ← l′, TLOC(|l′|) ← F , VAL(|l′|) ← 2d + (l′ & 1), Rl ← –l, F ← F + 1.” We do (∗∗) just before setting c ← c′ in answer 261. Furthermore, we set E ← F just after G ← 0 in step C1 and just after F ← F +1 in steps C6 and C9; and if G ≤ E after G ← G + 1 in step C4, we do (∗∗) with l0←l¯.

Answer 263 is modified in straightforward ways so that “clause Rl” is treated as if it were the binary clause (l∨l¯′) when Rl has the negative value –l′.

268. If MEM[c – 1] = k ≥ 3 is the size of clause c, and if 1 < j < k, we can delete the literal l in MEM[c + j] by setting k ← k – 1, MEM[c – 1] ← k, l′ ← MEM[c + k], MEM[c + j] ← l′, and MEM[c + k] ← l + f, where f is a flag (typically 2^31) that distinguishes a deleted literal from a normal one. (This operation does not need to be done when the current level d is zero; hence we can assume that k ≥ 3 and j > 1 before deletion. The flag is necessary so that global operations on the entire set of clauses, such as the purging algorithm, can pass safely over deleted literals. The final clause in MEM should be followed by 0, an element that’s known to be unflagged.)

269. (a) If the current clause contains a literal l=L¯t that is not in the trivial clause, where t is maximum, resolve the current clause with Rl¯ and repeat.

(b) (u¯1∨bj)∧(lj∨l¯j−1∨b¯j) for 1 ≤ j ≤ 9, (l0∨u¯2∨u¯3)∧(l¯9∨l¯8∨b¯10); l′ = l_0.

(c) If r ≥ d′ + τ , where τ is a positive parameter, learn the trivial clause instead of (l¯′∨b¯1∨…∨b¯r). (The watched literals should be l¯′ and ūd.)

Notice that this procedure will learn more than simple backtrack à la Algorithm D does, even when the trivial clause is always substituted (that is, even when τ = –∞), because it provides for backjumping when d′ < d +1.

270. (a) Consider the clauses 32¯, 43¯2¯, 54¯3¯1¯, 65¯4¯1¯, 6¯5¯4¯, with initial decisions L_1 ← 1, L_2 ← 2. Then L_3 ← 3 with reason R3←32¯; similarly L_4 ← 4, L_5 ← 5. If L_6 ← 6, the conflict clause 6¯5¯4¯ allows us to strengthen R_6 to 5¯4¯1¯; but if L6←6¯, with R6¯←6¯5¯4¯, we don’t notice that 65¯4¯1¯ can be strengthened. In either case we can, however, strengthen R_5 to 4¯3¯1¯, before learning the clause 2¯1¯.

(b) After doing blit (lj) to the literals of Rl, we know that Rl \ l is contained in {b¯1,…,b¯r} together with q + 1 unresolved false literals that have been stamped at level d. (Exercise 268 ensures that p ≠ 0 within each blit .) Thus we can subsume clause Rl on the fly if q + r +1 < k and q > 0.

In such cases the procedure of answer 268 can be used to delete l from c = Rl. But there’s a complication, because l = l_0 is a watched literal (j = 0 in that answer), and all other literals are false. After l is deleted, it will be essential to watch a false literal l′ that is defined at trail level d. So we find the largest j′ ≤ k such that VAL(MEM[c + j′]) ≥ 2d, and we set l′ ← MEM[c + j′]. If j′ ≠ k, we also set MEM[c + j′] ← MEM[c + k]; we can assume that j′ > 1. Finally, after setting MEM[c] ← l′ and MEM[c + k] ← l + f as in answer 268, we also delete c from the watch list Wl, and insert it into Wl′.

[This enhancement typically saves 1%–10% of the running time, but sometimes it saves a lot more. It was discovered in 2009, independently by two different groups of researchers: See H. Han and F. Somenzi, LNCS 5584 (2009), 209–222; Y. Hamadi, S. Jabbour, and L. Saïs, Int. Conf. Tools with Artif. Int. (ICTAI) 21 (2009), 328–335.]

271. We shall check for discards only if the current clause Ci is not trivial (see exercise 269), and if the first literal of Ci–1 does not appear in the trail. (Indeed, experience shows that almost every permissible discard falls into this category.) Thus, let Ci–1 be l_0l_1 ...lk–1 where VAL(|l_0|) < 0; we want to decide if {l¯′,b¯1,…,b¯r}⊆{l1,…,lk−1}.

The secret is to use the stamp fields that have already been set up. Set j ← k –1, q ← r + 1, and do the following while q > 0 and j ≥ q: If lj=l¯′, or if 0 ≤ VAL(|lj|) ≤ 2d′ + 1 and S(|lj|) = s, set q ← q – 1; in any case set j ← j – 1. Then discard if q = 0.

272. Reflection isn’t as easy to implement as it may seem, unless C is a unit clause, because CR must be placed carefully in MEM and it must be consistent with the trail. Furthermore, experience shows that it’s best not to learn the reflection of every learned clause, because excess clauses make unit propagation slower. The author has obtained encouraging results, however, by doing the following operations just before returning to C3 in step C9, whenever the length of C doesn’t exceed a given parameter R:

Assign ranks to the literals of CR by letting rank(l) = ∞ if l is on the trail, rank(l) = d″ if l¯ is on the trail at level d′ < d′, rank(l) = d otherwise. Let u and v be two of the highest ranking literals, with rank(u) ≥ rank(v). Put them into the first two positions of CR, so that they will be watched. Do nothing further if rank(v) > d′. Otherwise, if rank(v) < d′, backjump to level rank(v) and set d′ ← rank(v). Then if rank(u) = rank(v) = d′, treat CR as a conflict clause by going to step C7 with c ← CR. (That is a rare event, but it can happen.) Otherwise, if u doesn’t appear in the current trail, set LF ← u, TLOC(|u|) ← F , Ru ← CR, F ← F + 1. (Possibly F = E + 2 now.)

(For example, this method with R ← 6 roughly halved the running time of waerden (3, 10; 97) and waerden (3, 13; 160) with parameters (193) except for ρ ← .995.)

A similar idea works with the clauses langford (n), and in general whenever the input clauses have an automorphism of order 2.

273. (a) We can convert Algorithm C into a “clause learning machine” by keeping the process going after F reaches n in step C5: Instead of terminating, start over again by essentially going back to step C1, except that the current collection of clauses should be retained, and the OVAL polarities should be reset to random bits. Learned clauses of size K or less, where K is a parameter, should be written to a file. Stop when you’ve found a given number of short clauses, or when you’ve exceeded a given time limit.

For example, here’s what happened when the author first tried to find W (3, 13): Applying this algorithm to waerden (3, 13; 158) with K = 3, and with a timeout limit of 30 Gμ (gigamems), yielded the five clauses 65 68 70, 68 78 81, 78 81 90, 78 79 81, 79 81 82. So fifteen clauses 65 68 70, 66 69 71, ... , 81 83 84 could be added to waerden (3, 13; 160), as well as their fifteen reflections 96 93 91, 95 92 90, ... , 80 78 77. Then the algorithm “CR”of exercise 272 proved this augmented set unsatisfiable after an additional 107 Gμ. In a second experiment, using K = 2 with waerden (3, 13; 159) led to three binary clauses 76 84, 81 86, and 84 88. Shifting and reflecting gave twelve binary clauses, which in company with waerden (3, 13; 160) were refuted by CR in another 80 Gμ. (For comparison, Algorithm CR refuted waerden (3, 13; 160) unaided in about 120 Gμ, compared to about 270 Gμ for both Algorithm C and Algorithm L.) Optimum strategies for learning useful clauses from satisfiable subproblems are far from clear, especially because running times are highly variable. But this method does show promise, especially on more difficult problems — when more time can be devoted to the preliminary learning.

(b) Short clauses that can be learned from satisfiable instances of, say, X_0 → X_1 → … → Xr–1, when X_0 is not required to be an initial state, can be shifted and used to help refute X_0 → X_1 → … → Xr.

274. With care, circular reasoning can (and must) be avoided. But the author’s elaborate experiments with such ideas (and with the related notion of “better conflicts”) were disappointing; they didn’t beat the running time of the simpler algorithm. However, an intriguing idea by Allen Van Gelder [Journal on Satisfiability, Boolean Modeling and Computation 8 (2012), 117–122] shows promise.

275. When a solution has been found, let k be minimum such that xk = 1 and the value of xk has not been assigned at level 0. If no such k exists, we stop. Otherwise we are entitled to force variables x_1 through xk–1 all to have their current values, at level 0, because we know that this doesn’t produce an unsatisfiable problem. So we fix those values, and we restart the solution process at level 1 with the tentative decision ‘xk = 0’. If a conflict occurs, we’ll know that xk = 1 at level 0; if not, we’ll have a solution with xk = 0. In either case we can increase k. (This method is considerably better than that of answer 109, because every learned clause remains valid.)

276. True. Unit propagation essentially transforms F ∧ L into F | L.

277. Otherwise F ∧ C_1 ∧…∧ Ct–1 ⊢1 ∊ fails (unit propagation wouldn’t start).

278. For example, (46, 5¯6, 5¯4, 6, 4, ∊). (Six steps are necessary.)

279. True, because the dependency digraph contains a literal l with l→∗ l¯→∗l.

280. (a) They’re satisfied if and only if x_1 ...xn has at least j 0s and at least k 1s. [The problem cook (k, k) was introduced by Stephen A. Cook (unpublished) in 1971.]

(b) Take all positive (j – t)-clauses on {1,...,n – 1 – t} for t = 1, 2, ..., j.

(c) Suppose the very first decision is L_0 ← xn. The algorithm will proceed to act as if the input were cook (j, k) | xn = cook (j, k – 1). Furthermore, with these clauses, every clause that it learns initially will include x¯n. Therefore, by induction, the unit clause (x¯n) will be learned clause number (n−2j−1). All previously learned clauses are subsumed by this one, hence they’re no longer relevant. The remaining problem is cook (j,k)|x¯n=cook(j−1,k); so the algorithm will finish after learning (n−2j−2) more.

Similarly, if the first decision is L0←x¯n, the (n−2j−2) th learned clause will be (xn).

281. Stålmarck’s refutation corresponds to the sequence (Mjk1′, Mjk2′, ... , Mjk(k−1)′, Mj(k–1)) for j = 1, ... , k – 1, for k = m, m – 1, ... , 1. (Mjk(k−1)′ can be omitted.)

282. First learn the exclusion clauses (17). In the next clauses we shall write aj, bj, ..., as shorthand for aj,p, bj,p, ... , where p is a particular color, 1 ≤ p ≤ 3. Notice that the 12q edges appear in 4q triangles, namely {bj,cj,dj}, {aj,aj′ ,bj′ }, {fj,ej′ ,cj′}, {ej,fj′ ,dj′ }, for 1 ≤ j ≤ q, where j′ is j + 1 (modulo q). For every such triangle {u, v, w}, learn (ūp′ ∨ vp ∨ wp) and then (up ∨ vp ∨ wp), where p′ is p + 1 (modulo 3).

Now for j = 1,2, ... , q, learn (aj∨fj∨aj′ ∨ej′), (aj∨ej∨aj′ ∨fj′), (ej∨fj∨ej′ ∨fj′), (āj ∨ ēj ∨ ēj′), (a¯j∨f¯j∨f¯j′), (e¯j∨f¯j∨a¯j′), as well as eighteen more:

(u¯1∨v¯1∨uj′),(u¯2∨v¯2∨uj′′∨vj′′),if  j≥3  is  odd;(u¯1∨v¯1∨uj′),(u¯2∨v¯2∨u¯j′′),if  j≥3  is  even;
here u, v ∈ {a, e, f} and u′,v′ ∈ {a, e, f} yield 3 × 3 choices of (u, v, u′,v′). Then we’re ready to learn (āj ∨ ēj), (a¯j∨f¯j), (e¯j∨f¯j) for j ∈ {1, 2} and (aj ∨ ej ∨ fj ∨ aj′), (aj ∨ ej ∨ fj) for j ∈ {1,q}. All of these clauses are to be learned for 1 ≤ p ≤ 3.

Next, for j = q, q – 1, ... , 2, learn (āj ∨ ēj), (a¯j∨f¯j), (e¯j∨f¯j) for 1 ≤ p ≤ 3 and then (aj–1 ∨ ej–1 ∨ fj–1 ∨ aj), (aj–1 ∨ ej–1 ∨ fj–1) for 1 ≤ p ≤ 3. We have now established all clauses in the hint.

The endgame consists of the following for 1 ≤ p ≤ 3: For all choices of p′ and p″ with {p, p′,p″} = {1, 2, 3} (thus two choices), and for j = 2, 3, ... , q, learn three clauses

(a¯1,p∨e¯1,p′∨a¯j,p∨e¯j,p"),(a¯1,p∨e¯1,p′∨a¯j,p′∨ej,p),(a¯1,p∨e¯1,p′∨a¯j,p"∨ej,p′),j⁢  even;(a¯1,p∨e¯1,p′∨a¯j,p∨e¯1,p′),(a¯1,p∨e¯1,p′∨a¯j,p′∨ej,p"),(a¯1,p∨e¯1,p′∨a¯j,p"∨ej,p),j⁢  odd;
then learn (ā_1,p ∨ ē_1,p′). Finally learn ā_1,p.

[Not all of these clauses are actually necessary. For example, the exclusion clauses for b’s, c’s, and d’s aren’t used. This certificate doesn’t assume that the symmetry-breaking unit clauses b_1,1 ∧ c_1,2 ∧ d_1,3 of fsnark (q) are present; indeed, those clauses don’t help it much. The actual clauses learned by Algorithm C are considerably longer and somewhat chaotic (indeed mysterious); it’s hard to see just where an “aha” occurs!]

283. A related question is to ask whether the expected length of learned clauses is O(1) as q → ∞.

284. It’s convenient to represent each unit clause (l) in F ∪C_1 ∪...∪Ct as if it were the binary clause (l∨x¯0), where x_0 is a new variable that is always true. We borrow some of the data structures of Algorithm C, namely the trail array L, the reason array R, and the fields TLOC, S, VAL associated with each variable. We set VAL(k) = 0, 1, or –1 when xk has been forced true, forced false, or not forced, respectively.

To verify the clause Ci = (a_1 ∨...∨ ak), we begin with VAL(j) ← 0 for 0 ≤ j ≤ n, L_0 ← 0, L_1 ← ā_1, ... , Lk ← āk, E ← F ← k +1, G ← 0, and VAL(|Lp|) ← Lp & 1 for 0 ≤ p < F ; then we carry out unit propagation as in Algorithm C, expecting to reach a conflict before G = F . (Otherwise verification fails.)

A conflict arises when a clause c = l_0 ...lk–1 forces l_0 at a time when l¯0 has already been forced. Now we mimic step C7 (see exercise 263), but the operations are much simpler: Mark c, stamp S(|lj|) ← i for 0 ≤ j < k, and set p ← max (TLOC(|l_1|),..., TLOC(|lk–1|)). Now, while p ≥ E, we set l ← Lp, p ← p – 1, and if S(|l|) = i we also “resolve with the reason of l” as follows: Let clause Rl be l_0l_1 ...lk–1, mark Rl, and set S(|lj|) ← i for 1 ≤ j < k.

[Wetzler, Heule, and Hunt have suggested an interesting improvement, which will often mark significantly fewer clauses at the expense of a more complicated algorithm: Give preference to already-marked clauses when doing the unit propagations, just as Algorithm L prefers binary implications to the implications of longer clauses (see (62)).]

285. (a) j = 77, s_77 = 12 + 2827, m_77 = 59, b_77 = 710.

(b) j = 72, s_72 = 12 + 2048, m_72 = 99 + 243 + 404 + 536 = 1282, b_72 = 3 + 40 + 57 + 86 = 186. (The RANGE statistic is rather coarse when α=12, because many different signatures yield the same value.)

(c) j = 71, s_71 = 12 + 3087, m_71 = 243, b_71 = 40.

286. The maximum, 738, is achieved uniquely by the RANGE-oriented solution with α=1516, except that we can optionally include also the signatures (6, 0) and (7, 0) for which apq = 0. [This solution optimizes the worst case of clause selection, because the stated problem implicitly assumes that the secondary heuristic is bad. If we assume, however, that the choice of tie-breakers based on clause activity is at least as good as a random choice, then the expected number 738+45⋅1059≈745.6 from α=1516 is not as good as the expected number 710+287⋅57404≈750.5 from α=916.]

287. When a conflict is detected in step C7 (with d > 0), keep going as in step C3; but remember the first clause Cd that detected a conflict at each level d.

Eventually step C5 will find F = n. That’s when clauses get their RANGE scores, if we’re doing a full run because we want to purge some of them. (Sometimes, however, it’s also useful to do a few full runs at the very beginning, or just after a restart, because some valuable clauses might be learned.)

New clauses can be learned in the usual way from the remembered clauses Cd, in decreasing order of d, except that “trivial” clauses (exercise 269) are considered only at the lowest such level. We must keep track of the minimum backjump level d′, among all of these conflicts. And if several new clauses have the same d′, we must remember all of the literals that will be placed at the end of the trail after we eventually jump back.

288. Step C5 initiates a full run, then eventually finds F = n. At this point we’re done, in the unlikely event that no conflicts have arisen. Otherwise we set LS[d] ← 0 for 0 ≤ d < n and mj ← 0 for 1 ≤ j < 256. The activity ACT(c) of each learned clause c has been maintained in MEM[c – 5], as a 32-bit floating point number. The following steps compute RANGE(c), which will be stored in MEM[c – 4] as an integer, for all learned c in increasing order, assuming that c’s literals are l_0l_1 ...ls–1:

If Rl_0 = c, set RANGE(c) ← 0. Otherwise set p ← r ← 0, and do the following for 0 ≤ k < s: Set v ← VAL(|lk|). If v < 2 and v + lk is even, set RANGE(c) ← 256 and exit the loop on k (because c is permanently satisfied, hence useless). If v ≥ 2 and LS[lev (lk)] < c, set LS[lev (lk)] ← c and r ← r + 1. Then if v ≥ 2 and LS[lev (lk)] = c and lk + v is even, set LS[lev (lk)] ← c + 1 and p ← p + 1. After k reaches s, set r ← min (16(p + α(r – p)) , 255), RANGE(c) ← r, and mr ← mr +1.

Now resolve conflicts (see answer 287), giving ACT(c) ← 0 and RANGE(c) ← 0 to all newly learned clauses c, and jump back to trail level 0. (A round of purging is a major event, something like spring cleaning. It is possible that d′ = 0, in which case one or more literals have been appended to trail level 0 and their consequences have not yet been explored.) Find the median range j as defined in (124), where T is half the total current number of learned clauses. If j < 256 and T > sj, find h = T – sj clauses with RANGE(c) = j and ACT(c) as small as possible, and bump their range up to j +1. (This can be done by putting the first mj – h of them into a heap, then repeatedly bumping the least active as the remaining h are encountered; see exercise 6.1–22.)

Finally, go again through all the learned clauses c, in order of increasing c, ignoring c if RANGE(c) > j, otherwise copying it into a new location c′ ≤ c. (Permanently false literals, which are currently defined at level 0, can also be removed at this time; thus the clause’s size in MEM[c′ – 1] might be less than MEM[c – 1]. It is possible, but unlikely, that a learned clause becomes reduced to a unit in this way, or even that it becomes empty.) The activity score in MEM[c – 5] should be copied into MEM[c′ – 5]; but RANGE(c) and the watch links in MEM[c – 2] and MEM[c – 3] needn’t be copied.

When copying is complete, all the watch lists should be recomputed from scratch, as in answer 260, including original clauses as well as the learned clauses that remain.

289. By induction, yk = (2 – 21–k)Δ + (2(k – 2) + 22–k)δ for all k ≥ 0.

290. Set k ← HEAP[0]; then if VAL(k) ≥ 0, delete k from the heap as in answer 262, and repeat this loop.

291. OVAL(49) will be the even number 36, because of the propagations on level 18 that led to (115).

292. If AGILITY ≥ 2^32 – 2^13, then (127) either subtracts 2^19 – 1 or adds 1. Hence there’s a minuscule chance that AGILITY will overflow by passing from 2^32 – 1 to 2^32 (zero). (But overflow won’t be a calamity even if — unbelievably — it happens. So this is one “bug” in the author’s program that he will not try to fix.)

293. Maintain integers uf , vf, and θf , where θf has 64 bits. Initially uf = vf = Mf = 1. When M ≥ Mf in step C5, do this: Set Mf ← Mf + vf . If uf &–uf = vf, set uf ← uf + 1, vf ← 1, θf ← 2^32ψ; otherwise set vf ← 2vf and θf ← θf + (θf ≫4). Flush if AGILITY ≤ θf.

294. We have, for example, g1100=z3(g0100+g1000+g1110), and g01∗1 = 1. The solution is g00∗1 = g01∗0 = g11∗1 = A/D, g00∗0 = g10∗1 = g11∗0 = B/D, g10∗0 = C/D, where A = 3z – z^2 – z^3, B = z^2, C = z^3, D = 9 – 6z – 3z^2 + z^3. Hence the overall generating function is g = (6A +6B +2C +2D)/(16D); and we find g′(1) = 33/4, g″(1) = 147. Thus mean(g) = 8.25, var(g) = 87.1875, and the standard deviation is ≈ 9.3.

295. Consider all 3(n3) clauses x¯i∨xj∨xk for distinct {i, j, k}, plus two additional clauses (x¯1∨x¯2∨x¯3)∧(x¯4∨x¯5∨x¯6) to make the solution 0 ... 0 unique. Only the two latter clauses cause the variables Xt and Yt in the proof of Theorem U to deviate from each other. [C. Papadimitriou, Computational Complexity (1994), Problem 11.5.6. These clauses spell trouble for a lot of other SAT algorithms too.]

296. The hinted ratio 2(2p+q +1)(2p+q)/(9(p+1)(p+q +1)) is ≈ 1 when p ≈ q (more precisely when p = q – 7+ O(1/q)). And f(q+1,q+1)/f(q,q)=2(n−q)(3q+3)3¯/(27(q+1)2(2q+2)2¯) is ≈ 1 when q ≈ n/3. Finally, f(n/3,n/3)=34πn(3/4)n(1+O(1/n)) by Stirling’s approximation, when n = 3q.

297. (a) Gq(z) = (z/3)qC(2z^2/9)q = G(z)q where G(z)=(3−9−8z2)/(4z), by Eqs. 7.2.1.6–(18) and (24). [See Algorithmica 32 (2002), 620–622.]

(b) Gq(1) = 2–q is the probability that Yt actually reaches 0, for some finite t

(c) If the Y process does stop, Gq(z)/Gq(1) = (2G(z))q describes the distribution of stopping times. Hence Gq′(1)/Gq(1)=2qG′(1)=3q is the mean length of the random walk, given that it terminates. (The variance, incidentally, is 24q. A random Y -walker who doesn’t finish quickly is probably doomed to wander forever.)

(d) The generating function for T , the stopping time of the Y process, is T(z)=Σq(nq)2−nGq(z); and T is finite with probability T(1)=(34)n by (b). If we restrict consideration to such scenarios, the mean T′(1)/T (1) is n; and Markov’s inequality tells us that Pr(T ≥ N | the algorithm terminates) ≤ n/N.

(e) The algorithm succeeds with probability p > Pr(T < N) ≥ (1 – n/N)(3/4)n, when it is given satisfiable clauses. So it fails after K(4/3)n trials with probability less than exp(K(4/3)n ln(1 – p)) < exp(–K(4/3)np) < exp(–K/2) when N = 2n.

298. Change 1/3 and 2/3 in (129) to 1/k and (k – 1)/k. The effect is to change G(z) to (z/k)C((k – 1)z^2/k^2), with G(1) = 1/(k – 1) and G′(1) = k/((k – 1)(k – 2)). As before, T (1) = 2–n(1 + G(1))n and T′(1)/T (1) = nG′(1)/(1 + G(1)). So the generalized Corollary W gives success probability > 1 – e–K/2 when we apply Algorithm P K(2 – 2/k)n times with N = ⌊2n/(k – 2)⌋.

299. In this case G(z)=(1−1−z2)/z; thus G(1) = T (1) = 1. But G′(1) = ∞, so we must use a different method. The probability of failure if N = n^2 is

12nΣp,q(nq)q2p+q(2p+qp)[2p+q>n2]22p+q=Σt>n22−n−ttΣp(nt−2p)(tp)(t−2p)≤Σt>n22−n−tt(t⌊t/2⌋)    Σp(nt−2p)(t−2p)=n4Σt>n22−tt(t⌊t/2⌋)<n4Σt>n2  2πt3=n8π ∫n2∞  dx⌈x⌉3/2<n8π∫n2∞  dxx3/2=12π.

[See C. Papadimitriou, Computational Complexity (1994), Theorem 11.1.]

300. In this algorithm, variables named with uppercase letters (except C and N) denote bit vectors of some fixed size (say 64); each bit position represents a separate trial. The notation Ur stands for a vector of random bits, each of which is 1 with probability 1/r, independently of all other bits and all previous U’s. The maximum number of flips per bit position in this variant of Algorithm P is only approximately equal to N.

P1′. [Initialize.] Set Xi ← U_2 for 1 ≤ i ≤ n. Also set t ← 0.

P2′. [Begin pass.] Set Z ← 0 and j ← 0. (Flipped positions are remembered in Z.)

P3′. [Move to next clause.] If j = m, go to P5′. Otherwise set j ← j +1.

P4′. [Flip.] Let Cj be the clause (l_1 ∨… ∨ lk). Set Y←L¯1  &…&L¯k, where Li denotes Xh if li = xh and Li denotes X¯h if li=x¯h. (Thus Y has 1s in positions that violate clause Cj.) Set Z ← Z | Y and t ← t +(Y & 1). Then for r = k, k – 1, ... , 2 set Y′ ← Y & Ur, Lr ← Lr ⊕ Y′, Y ← Y – Y′. Finally set L_1 ← L_1 ⊕ Y and return to P3′.

P5′. [Done?] If Z ≠ –1, terminate successfully: One solution is given by the bits (X_1 & B) ... (Xn & B), where B=Z¯  &  (Z+1). Otherwise, if t > N, terminate unsuccessfully. Otherwise return to P2′.

The shenanigans in step P4′ have the effect of flipping the offending bits of each literal with probability 1/k, thus distributing the 1s of Y in an unbiased fashion.

301. In practice we can assume that all clauses have limited size, so that (say) k ≤ 4 in step P4′. The clauses can also be sorted by size.

A traditional random number generator produces bits U_2; and one can use U_2&U_2 to get U_4. The method of exercise 3.4.1–25 can be used for other cases; for example,

U2  &(U2 | (U2  &(U2 | (U2  &(U2 |  (U2 & (U2 |  (U2  & U2))) )))))

is a sufficiently close approximation to U_3. The random numbers needed in step P1′must be of top quality; but those used in step P4′ don’t have to be especially accurate, because most of their bits are irrelevant. We can precompute the latter, making tables of 2d values for each of U_2, U_3, U_4, and running through them cyclically by means of table indices U2P, U3P, U4P as in the code below, where UMASK = 2d+3 – 1. The values of U2P, U3P, and U4P should be initialized to (truly) random bits whenever step P2′ starts a new pass over the clauses.

Here is sample code for the inner loop, step P4′, for clauses with k = 3. The octabyte in memory location L +8(i–1) is the address in memory where Xh is stored, plus 1 if it should be complemented; for example, if l_2 is x¯3, the address X+3×8+1 will be in location L + 8, where L is a global register. Register mone holds the constant –1.

LDOU $1,L,0 addr(L_1)

XOR $9,$6,$0 L¯3

STOU $6,$3,0 |L_3|⊕ Y′

LDOU $4,$1,0 |L_1|

AND $7,$7,$8

SUBU $7,$7,$0

LDOU $2,L,8 addr(L_2)

AND $7,$7,$9 Y

LDOU $0,U2,U2P

LDOU $5,$2,0 |L_2|

OR Z,Z,$7 Z | Y

ADD U2P,U2P,8

LDOU $3,L,16 addr(L_3)

AND $0,$7,1 Y &1

AND U2P,U2P,UMASK

LDOU $6,$3,0 |L_3|

ADD T,T,$0 new t

AND $0,$0,$7 U_2 & Y

ZSEV $0,$1,mone

LDOU $0,U3,U3P

XOR $5,$5,$0

XOR $7,$4,$0 L¯1

ADD U3P,U3P,8

STOU $5,$2,0 |L_2|⊕ Y′

ZSEV $0,$2,mone

AND U3P,U3P,UMASK

SUBU $7,$7,$0

XOR $8,$5,$0 L¯2

AND $0,$0,$7 U_3 & Y

XOR $4,$4,$7

ZSEV $0,$3,mone

XOR $6,$6,$0

STOU $4,$1,0 |L_1|⊕ Y

302. Assume that literals are represented internally as in Algorithm A, and that all clauses have strictly distinct literals. An efficient implementation actually requires more arrays than are stated in the text: We need to know exactly which clauses contain any given literal, just as we need to know the literals of any given clause.

W4. [Choose l.] Set g ← [U ≥ p], c ← ∞, j ← z ← 0, and do the following while j < k: Set j ← j + 1. Then if c|lj| < c and either c|lj| = 0 or g = 1, set c ← c|lj| and z ← 0. Then if c|lj| ≤ c, set z ← z + 1, and if zU < 1 also set l ← lj. (Here each random fraction U should be independent of the others.)

W5. [Flip l.] Set s ← 0. For each j such that Cj contains l, make clause Cj happier as follows: Set q ← kj, kj ← q +1; and if q = 0, set s ← s+1 and delete Cj from the f array (see below); or if q = 1, decrease the cost of Cj’s critical variable (see below). Then set c|l| ← s and x|l|←x¯|l|. For each j such that Cj contains l¯, make clause Cj sadder as follows: Set q ← kj –1, kj ← q; and if q = 0, insert Cj into the f array (see below); or if q = 1, increase the cost of Cj’s critical variable (see below). Set t ← t + 1 and return to W2.

To insert Cj into f, we set fr ← j, wj ← r, and r ← r + 1 (as in step W1). To delete it, we set h ← wj, r ← r – 1, fh ← fr, wfr ← h.

Whenever we want to update the cost of Cj’s critical variable in step W5, we know that Cj has exactly one true literal. Thus, if the literals of Cj appear sequentially in a master array M, it’s easy to locate the critical variable x|Mi|: We simply set i ← START(j); then while Mi is false (namely while x|Mi| = Mi & 1), set i ← i +1.

A slight refinement is advantageous when we will be increasing c|Mi|: If i ≠ START(j), swap MSTART(j) ↔ Mi. This change significantly shortens the search when c|Mi| is subsequently decreased. (In fact, it reduced the total running time by more than 5% in the author’s experiments with random 3SAT problems.)

303. In this case D = 3 – z – z^2 = A/z, and we have g′(1) = 3, g″(1) = 73/4. Thus mean(g) = 3 and var(g) = 12.25 = 3.5^2.

304. If νx = x_1 + … + xn = a, there are a(n – a) unsatisfied clauses; hence there are two solutions, 0 ... 0 and 1 ... 1. If x_1 ...xn isn’t a solution, Algorithm P will change a to a ± 1, each with probability 12. Thus the probability generating function ga for future flips is 1 when a = 0 or a = n, otherwise it is z(ga–1 + ga+1)/2. And the overall generating function is g=Σa    (na)ga/2n. Clearly ga = gn–a.

Exercise MPR–105 determines ga and proves that the mean number of flips, ga′(1), is a(n – a) for 0 ≤ a ≤ n. Thus g′(1)=2−n  ∑a=0n(na)ga′(1)=12(n2).

Turning now to Algorithm W, again with x_1 + … + xn = a, the cost of xi is a – 1 when xi = 1, n – a – 1 when xi = 0. Therefore g_1 = gn–1 = z in this case. And for 2 ≤ a ≤ n – 2, a ≠ n/2, we will move closer to a solution with probability q and farther from a solution with probability p, where p + q = 1 and p = p′/2 ≤ 1/2; here p′ is the greed-avoidance parameter of Algorithm W. Thus for 2 ≤ a ≤ n/2 we have ga = gn–a = z(qga–1 + pga+1).

If p′ = 0, so that the walk is 100% greedy, Algorithm W zooms in on the solution, with ga = za. Exercise 1.2.6–68 with p = 1/2 tells us that g′(1)=n/2−m(nm)/2n=n/2−n/2π+O(1) in that case. On the other hand if p′ = 1, so that the walk is greedy only when a = 1 or a = n – 1, we’re almost in the situation of Algorithm P but with n decreased by 2. Then g′(1)=2−n   ∑a=1n−1(na)(1+(a−1)(n−2)−(a−1)2)=n(n−5)/4+2+(2n−4)/2n  ; greed triumphs.

What happens as p′ rises from 0 to 1? Let’s decrease n by 2 and use the rule ga = z(qga–1 + pga+1) for 1 ≤ a ≤ n/2, so that the calculations resemble those we did for Algorithm P but with p now ≤ 1/2 instead of p = 1/2. Functions tk and uk can be defined as in MPR–105; but the new recurrences are tk+1 = (tk – pz^2tk–1)/q and uk+1 = (uk – pz^2uk–1)/q. Hence

T(w)=q−pwq−w+pz2w2;U(w)=q−(1−qz)wq−w+pz2w2.

Differentiating with respect to z, then setting z = 1, now yields
tk′(1)=2pq(1−(p/q)k)(q−p)2−2pkq−p,uk′(1)=2pq(1−(p/q)k)q(q−p)2−2p(k−1/2)q−p.

It follows that ga′(1)=a/(q−p)−2pq((p/q)m−a−(p/q)m/(q−p)2 for 0 ≤ a ≤ n/2 when n is even, a/(q – p) – q((p/q)m–a – (p/q)m)/(q – p)2 when n is odd. The overall totals when n = 1000 and p′ = (.001, .01, .1, .5, .9, .99, .999) are respectively ≈ (487.9, 492.3, 541.4, 973.7, 4853.4, 44688.2, 183063.4).

305. That little additional clause reverses the picture! Now there’s only one solution, and greediness fails badly when νx > n/2 because it keeps trying to move x away from the solution. To analyze the new situation in detail, we need 3(n – 1) generating functions gab, where a = x_1 + x_2 and b = x_3 + … + xn. The expected number of flips will be g′(1), where g=2−n  Σb=02 Σb=0n−2   (2a)(n−2b)gab.

The behavior of Algorithm P is ambiguous, because the unsatisfied clause found in step P2 depends on the clause ordering. The most favorable case arises when a = 2, because we can decrease a to 1 by working on the special clause x¯1∨x¯2. Any other clause is equally likely to increase or decrease a + b. So the best-case generating functions maximize the chance of reaching a = 2: g_00 = 1, g01=z2(g00+g11), g02=z2(g01+g12), g10=z2(g00+g20), g11=z2(g10+g21), g12=z2(g11+g22), and g_2b = zg_1b. The solution has g_1b = (z/(2 – z^2))b+1; and we find mean(g) = 183/32 = 5.71875.

The worst case arises whenever g_20 ≠ zg_10 and g_21 ≠ zg_11; for example we can take g20=z2(g10+g21), g21=z2(g20+g22), together with the other seven equations from the best case. Then g_01 = g_10 = z(4–3z^2)/d, g_02 = g_11 = g_20 = z^2(2–z^2)/d, and g_12 = g_21 = z^3/d, where d = 8–8z^2 +z^4. Overall, g = (1+z)2(2–z^2)/(4d) and mean(g) = 11.

(This analysis can be extended to larger n: The worst case turns out to have gab = ga+b = (z/2)a+b tn–a–b/tn, in the notation of the previous exercise, giving n(3n – 1)/4 flips on average. The best case has g_1b as before; hence g0b′=3b+2−21−b, g16′=3b+3, and g2b′=3b+4 when z = 1. The best average number of flips therefore turns out to be linear, with mean (g)=32n−89(3/4)n.)

The analysis becomes more exciting, but trickier, when we use Algorithm W. Let p = p′/2 and q = 1 – p as in the previous answer. Clearly g_00 = 1, g_01 = g_10 = zg_00, g02=z2(g01+g12), and g_22 = zg_12; but the other four cases need some thought. We have
g11=z4((12+q)(g01+g10)+g12+2pg21),
since the costs for x_1x_2x_3x_4 = 1010 are 1211 and the unsatisfied clauses are (x¯1∨x4), (x¯3∨x4), (x¯1∨x2), (x¯3∨x2); in the former two clauses we flip each literal equally often, but in the latter two we flip x_2 with probability p and the other with probability q. A similar but simpler analysis shows that g21=z4(g11+3g22) and g20=z5(g10+2g21).

The most interesting case is g12=z3(pg02+2pg11+3qg22), where the costs are 2122 and the problematic clauses are (x¯1∨x2), (x¯3∨x2), (x¯4∨x2). If p = 0, Algorithm W will always decide to flip x_2; but then we’ll be back in state 12 after the next flip.

Indeed, setting p = 0 yields g_00 = 1, g_01 = g_10 = z, g02=12z2, g11=34z2, g20=35z2+340z4, g21=316z3, and g_12 = g_22 = 0. The weighted total therefore turns out to be g = (40 + 160z + 164z^2 +15z^3 +3z^4)/640. Notice that the greedy random walk never succeeds after making more than 4 flips, in this case; so we should set N = 4 and restart after each failure. The probability of success is g(1) = 191/320. (This strategy is actually quite good: It succeeds after making an average of 1577/382 ≈ 4.13 flips and choosing random starting values x_1x_2x_3x_4 about 320/191 times.)

If p is positive, no matter how tiny, the success probability for N = ∞ is g(1) = 1. But the denominator of g is 48 – 48z^2 + 26pz^2 + 6pz^4 – 17p^2z^4, and we find that mean(g) = (1548+2399p–255p^2)/(1280p–680p^2) = (6192+4798p′–255p′2)/(2560p′ – 680p′2 Taking p′ = (.001, .01, .1, .5, .9, .99, .999) in this formula gives, respectively, the approximate values (2421.3, 244.4, 26.8, 7.7, 5.9, 5.7, 5.7).

(Calculations for n = 12 show that g is a polynomial of degree 8 when p = 0, with g(1) ≈ .51 and g′(1) ≈ 2.40. Thus, setting N = 8 yields success after about 16.1 flips and 1.95 initializations. When p > 0 we have g′(1) ≈ 1.635p–5 + O(p–4) as p → 0, and the seven values of p′ considered above yield respectively (5 × 10^16, 5 × 10^11, 5 × 10^6, 1034.3, 91.1, 83.89, 83.95) flips—surprisingly not monotone decreasing in p′. These WalkSAT statistics can be compared with 17.97 to 105 flips for Algorithm P.)

306. (a) Since l(N) = EN + (1 – qN)(N + l(N)), we have qNl(N) = EN + N – NqN = p_1 + 2p_2 + ... + NpN + NpN+1 + ... + Np∞ = N – (q_1 + ... + qN–1).

(b) If N = m + k and k ≥ 0 we have EN = m^2/n, q_1 + … + qN–1 = km/n, and qN = m/n; hence l(N) = n + k(n – m)/m.

(c) If N ≤ n, l(N)=(N−(N2)/n)/(N/n)=n−N−12; otherwise l(N)=l(n)=n+12.

(d) From qN = p_1(N – q_1 –…– qN–1) and qN+1 = p_1(N +1 – q_1 – … – qN ) we deduce pN+1 = p_1(1 – qN ), hence 1 – qN+1 = (1 – p_1)(1 – qN ). So it’s a geometric distribution, with pt = p(1 – p)t–1 for t ≥ 1. (The fact that l(1) = l(2) = … is called the “memoryless property” of the geometric distribution.)

(e) Choose p_1, ... , pn arbitrarily, with qn = p_1 + … + pn ≤ 1. Then, arguing as in (d), pn+1, pn+2, ... are defined by 1 – qN = (1 – 1/l(n))N–n(1 – qn) for N ≥ n.

(f) Since l(n +1) – l(n) = (n – (q_1+ … +qn))(1 – 1/qn) ≤ 0, we must have qn = 1 and l(n) = l(n + 1). (The case l(n) < l(n + 1) is impossible.)

(g) Let x = p_1 and y = p_2. By part (f), the conditions are equivalent to 0 < x ≤ x + y < 1 and x(3 – 2x – y) > 1. Hence 0 < (2x–1)(1–x)–xy ≤ (2x–1)(1–x); we get the general solution by first choosing 12<x<1, then 0 ≤ y < (2x – 1)(1 – x)/x.

(h) If N∗ = ∞ and l(n) < ∞, we can find n′ with qn′l(n′) = p_1+2p_2+…+n′pn′ + n′pn′+1 + … + n′p∞ > l(n). Hence l(N) ≥ qN l(N) ≥ qn′l(n′) > l(n) for all N ≥ n′.

(i) We have qn+k = k/(k + 1) for k ≥ 0; hence l(n + k) = (k + 1)(n + Hk)/k. The minimum occurs when l(n + k) ≈ l(n + k – 1), namely when n ≈ k – Hk; thus k = n +ln n + O(1). For example, the optimum cutoff value when n = 10 is N∗ = 23. (Notice that E∞ = ∞, yet l = l(N∗) ≈ 14.194 in this case.)

(j) Let pt = [t> 1]/2t–1. Then l(N) = (3 – 22–N)/(1 – 21–N) decreases to 3.

(k) Clearly l ≤ L. For N ≤ L we have l(N) = (N – (q_1 + … + qN–1))/qN ≥ (N –(1+ … +(N–1))/L)/(N/L) = L–(N –1)/2 ≥ (L+1)/2. And for N = ⌊L⌋ +k+1, similarly, l(N) ≥ N – (1+ … + ⌊L⌋ + kL)/L = ⌊L + 1⌋ (1 – ⌊L⌋ /(2L)) ≥ (L +1)/2.

307. (a) E X = EN_1 +(1 – qN_1)(N_1 +E X′), where X′ is the number of steps for the sequence (N_2,N_3,... ). For numerical results, start with j ← 0, s ← 0, α ← 1; then, while α > ∊, set j ← j +1, α ← (1 – qNj)α, and s ← s + ENj + αNj. (Here ∊ is tiny.)

(b) Let Pj = (1 – qN_1) ... (1 – qNj–1) = Pr(X > Tj), and note that Pj ≤ (1 – pn)j–1 where n = min{t | pt > 0}. Since qN l(N) = EN +(1 – qN )N, we have

EX=qN1l(N1)+(1−qN1)(qN2l(N2)+(1−qN2)(qN3l(N3)+…))                                                                                                                              =Σj=1∞PjqNjl(Nj)=Σj=1∞(Pj−Pj+1)l(Nj).

(c) EX≥Σj=1∞(Pj−Pj+1)l(N∗)=l.

(d) We can assume that Nj ≤ n for all j; otherwise the strategy would do even worse. For the hint, let {N_1,...,Nr} contain rm occurrences of m, for 1 ≤ m ≤ n, and suppose tm = rm + … + rn. If tm < n/(2m), the probability of failure would be (1 – m/n)tm ≥ 1 – tmm/n > 1/2. Hence we have tm ≥ n/(2m) for all m, and N_1 + … + Nr = t_1 + … + tn ≥ nHn/2.

Now there’s some m such that the first r – 1 trials fail on p(m) with probability > 12. For this m we have EX>12(N1+…+Nr−1)≥12(N1+…+Nr−n).

308. (a) 2a+1 – 1; and we also have S2a+b = Sb+1 for 0 ≤ b < 2a – 1 (by induction).

(b) The sequence (un,vn) in (131) has 1 + ρk entries with un = k; and ρ1 + … + ρn = n – νn by Eq. 7.1.3–(61). From the double generating function g(w, z) = Σn≥0 wνnzn = (1+wz)(1+wz^2)(1+wz^4)(1+wz^8) ... we deduce that Σk≥0 z2k+1–νk = zg(z–1,z^2).

(c) {n |Sn = 2a} = {2a+1k +2a+1 – 1 – νk | k ≥ 0}; hence Σn≥0 zn[Sn = 2a] = z2a+1–1g(z–1,z2a+1) = z2a+1–1 (1 + z2a+1–1) (1 + z2a+1–1) (1 + z2a+3–1)....

(d) When 2a occurs for the 2bth time, we’ve had 2a+b–c – [c>a] occurrences of 2c, for 0 ≤ c ≤ a + b. Consequently Σ(a, b, 1) = (a + b – 1)2a+b + 2a+1.

(e) The exact value is Σc=0a+b 2a+b−c2c+Σc=1ρk2a+b+c ; and ρk ≤ λk = [lg k].

(f) The stated formula is E mink {Σ(a, b, k) | Σ(a, b, k) ≥ X}, if we penalize the algorithm so that it never succeeds unless it is run with the particular cutoff N = 2a.

(g) We have Q ≤ (1 – qt)2b ≤ (1 – qt)1/qt < e–1; hence E X < (a + b – 1)2a+b + 2a+1+Σk=1∞(a+b+2k−1)2a+be−k = 2a+b((a + b)e/(e – 1) + e(3 – e)/(e – 1)2 +21–b). Furthermore we have 2a+b < 8l – 4l[b = 0], by exercise 306(k).

309. No — far from it. If Algorithm C were to satisfy the hypotheses of exercise 306, it would have to do complete restarts: It would not only have to flush every literal from the trail, it would also have to forget all the clauses that it has learned, and reinitialize the random heap. [But reluctant doubling appears to work well also outside of Vegas.]

310. A method analogous to (131) can be used: Let (u1′,v1′)=(1,0); then define (un+1′,vn+1′)=(un′⁢ &−un′=1≪vn′? (succ(un′), 0): (un′,vn′+1)). Here ‘succ’ is the Fibonacci-code successor function that is defined by six bitwise operations in answer 7.1.3–158. Finally, let Sn′=Fvn′+2 for n ≥ 1. (This sequence 〈Sn′〉, like 〈Sn〉, is “nicely balanced”; hence it is universal as in exercise 308. For example, when Fa appears for the first time, there have been exactly Fa+2–c occurrences of Fc, for 2 ≤ c ≤ a.)

311. Because 〈Rn〉 does surprisingly well in these tests, it seems desirable to consider also its Fibonacci analog: If fn = succ(fn–1) is the binary Fibonacci code for n, we can call 〈ρ′n〉 = 〈ρfn〉 = (0, 1, 2, 0, 3, 0, 1, 4, 0,... ) the “Fibonacci ruler function,” and let 〈Rn′〉=(1,2,3,1,5,1,2,8,1,…)) be the “ruler of Fibonaccis,” where Rn′=F2+ρ′n.

The results (ES,ES′ ,ER,ER′) for m = 1 and m = 2 are respectively (315.1, 357.8, 405.8, 502.5) and (322.8, 284.1, 404.9, 390.0); thus S beats S′ beats R beats R′ when m = 1, while S′ beats S beats R′ beats R when m = 2. The situation is, however, reversed for larger values of m: R beats R′ beats S beats S′ when m = 90, while R′beats R beats S′ beats S when m = 89.

In general, the reluctant methods shine for small m, while the more “aggressive” ruler methods forge ahead as m grows: When n = 100, S beats R if and only if m ≤ 13, and S′ beats R′ if and only if m ≤ 12. The doubling methods are best when m isapower of 2 or slightly less; the Fibonacci methods are best when m is a Fibonacci number or slightly less. The worst cases occur at m = 65 = 2^6 + 1 for S and R (namely 1402.2 and 845.0); they occur at m = 90 = F_11 + 1 for S′ and R′ (namely 1884.8 and 805.9).

312. T (m, n) = m + b2bh_0(θ)/θ + 2bg(θ), where b = ⌈lg m⌉, θ = 1 – m/n, ha(z) = Σn zn[Sn =2a], and g(z) = Σn≥1 Snzn = Σa≥0 2aha(z).

313. If l is flipped, the number of unsatisfied clauses increases by the cost of |l| and decreases by the number of unsatisfied clauses that contain l; and the latter is at least 1.

Consider the following interesting clauses, which have the unique solution 0000:

x1∨x¯2,  x¯1∨x2,  x2∨x¯3,  x¯2∨x3,  x3∨x¯4,  x¯3∨x4,  x¯1∨x¯4.

“Uphill” moves 1011 ↦ 1111 and 1101 ↦ 1111 are forced; also 0110 ↦ 1110 or 0111.

314. (Solution by Bram Cohen, 2012.) Consider the 10 clauses 1¯2¯345¯67, 1¯23¯45¯67, 1234¯5, 1234¯6, 1234¯7, 1¯2¯3¯4¯, 1¯2¯3¯5¯, 1¯2¯3¯6¯, 1¯2¯4¯5¯, 1¯2¯4¯6¯, and 60 more obtained by the cyclic permutation (1234567). All binary x = x_1 ...x_7 with weight νx = 2 have cost-free flips leading to weight 3, but no such flips to weight 1. Since the only solution has weight 0, Algorithm W loops forever whenever νx > 1. (Is there a smaller example?)

315. Any value with 0 ≤ p < 1/2 works, since each graph component is either K_1 or K_2.

316. No; max θ(1 – θ)d for 0 ≤ θ < 1 is dd/(d + 1)d+1, when θ = 1/(d + 1). [But Theorem J for d > 2 is a consequence of the improved Theorem L in exercise 356(c).]

317. Number the vertices so that the neighbors of vertex 1 are 2, ... , d′, and let Gj = G \ {1,...,j}. Then α(G) = α(G_1) – Pr(A_1 ∩ Ā_2 ∩ … ∩ Ām), and the latter probability is ≤ Pr(A_1∩Ād′+1 ∩…∩Ām) = Pr(A_1 | Ād′+1 ∩…∩Ām)α(Gd′) ≤ pα(Gd′).

Let ρ = (d – 1)/d. By induction we have α(Gj) > ρα(Gj+1) for 1 ≤ j < d′, because vertex j + 1 has degree < d in Gj. If d′ = 1 then α(G) ≥ α(G_1) – ρα(G_1) > ρα(G_1) > 0. Otherwise if d′ ≤ d, α(G) ≥ α(G_1) – pα(Gd′) > α(G_1) – pρ1–d′ α(G_1) ≥ α(G_1) – pρ1–d α(G_1) = ρα(G_1) > 0. Otherwise we must have d′ = d + 1, with vertex 1 of degree d, and α(G)>α(G1)−pρ−dα(G1)=d−2d−1α(G1)≥0.

318. Let fn = MG(p) where G is the graph of a complete t-ary tree with tn leaves; thus G has tk vertices at distance k from the root, for 0 ≤ k ≤ n. Then
f0=1−p,f1=(1−p)t−p,andfn+1=fnt−pfn−1t2⁢  for⁢ n>1.⁢

By Theorem S, it suffices to show that fn ≤ 0 for some n.

The key idea is to let g_0 = 1 – p and gn+1=fn+1/fnt=1−p/gnt. Assuming that gn > 0 for all n, we have g_1 < g_0 and gn−gn+1=p/gnt−p/gn+1t>0 when gn + 1 < gn. Hence limn→∞ gn = λ exists, with 0 < λ < 1. Furthermore λ = 1 – p/λt, so that p = λt(1 – λ). But then p ≤ tt/(t + 1)t + 1 (see answer 316 with θ = 1 – λ).

[One must admit, however, that the limit is not often reached until n is extremely large. For example, even if t = 2 and p = .149, we don’t have fn < 0 until n = 45. Thus G must have at least 2^45 vertices before this value of p is too large for Lemma L.]

319. Let x = 1/(d – 1). Since ex > 1 + x = d/(d – 1), we have e > (d/(d – 1))d-1.

320. (a) Let fm(p) be the Möbius polynomial when p_1 = … = pm = p. Then we have fm(p) = fm–1(p) – pfm–2(p), and one can show by induction that fm(1/(4 cos^2 θ)) = sin((m + 2)θ)/((2 cos Ν)m+1 sin θ). The threshold decreases to 1/4 as m → ∞.

(b) 1/(4⁢ cos2⁢ π2m); the Möbius polynomial gm(p) = fm–1(p) – pfm–3(p) satisfies the same recurrence as fm(p), and equals 2 cos mθ/(2 cos θ)m when p = 1/(4 cos^2 θ).

[In terms of the classical Chebyshev polynomials, gm(p)=2pm/2Tm(1/2p)) and fm(p)=p(m+2)/2Um+1(1/2p)).]

321. Let θ=(2−2)/2, θ′=θ(1−θ)=(2−1)/2, and c = (p – θ)/(1 – θ). The method of answer 345 gives (Pr(A¯B¯C¯D¯), Pr(AB¯C¯D¯), Pr(ABC¯D¯), Pr(AB¯CD¯), Pr(ABCD¯), Pr(ABCD)) = (0, θ′(1–c)3, 2θ′(1–c)c, θ^2(1–c)2+2θ′(1–c)3, θ^2(1–c)c+3θ′(1–c)c^2, θ^2c^2 + 4θ′c^3). Other cases are symmetric to these six. When p = 3/10 the six probabilities are ≈ (0, .20092, .00408, .08815, .00092, .00002).

322. (a) Let aj = Σiwi⌈ij∈A⌉, bj = Σkyk⌈jk∈B⌉, cl = Σkyk⌈kl∈C⌉, and dl = Σiwi⌈li∈D⌉. Then when X = j and Z = l, the best way to allocate the events is

within W and Y. Hence Pr(A¯∩B¯∩C¯∩D¯)=Σj,l⁢ xjzl ((a¯j+d¯l)−1) ((b¯j+c¯l)−1), which is zero if and only if we have aj + dl ≥ 1 or bj + cl ≥ 1 for all j and l with xjzl > 0.

(b) Since Σjxj(aj, bj) = (p, p), the point (p, p) lies in the convex hull of the points (aj, bj). So there must be points (a, b) = (aj′, bj′) and (a′, b′) = (aj′, bj′) such that the line from (a, b) to (a′, b′) intersects the region {(x, y) | 0 ≤ x, y ≤ p}; in other words μa + (1 – μ)a′ ≤ p and μb + (1 – μ)b′ ≤ p. Similarly we can find c, d, c′, d′, ν.

(c) Fact: If a≥23 and b′≥23, then μ≥12; hence a=b′=23 and a′ = b = 0. Notice also that there are 16 symmetries, generated by (i) a ↔ b, c ↔ d; (ii) a ↔ a′, b ↔ b′, μ ↔ 1 – μ; (iii) c ↔ c′, d ↔ d′, ν ↔ 1 – ν; (iv) a ↔ d, b ↔ c, μ ↔ ν.

If c ≤ c′ and d ≤ d′, or if c≤13 and d≤13, we can assume (by symmetry) that the Fact applies; this gives a solution to all the constraints, with c=d=c′=d′=13.

For the remaining solutions we may assume that a, b′>13>a′,b. Suppose the line from (a, b) to (a′,b′) intersects the line from (0, 0) to (1, 1) at the point (α, α); dividing a, b, a′, b′ by 3α gives a solution in which μa+(1−μ)a′=μb+(1−μ)b′=13. Similarly, we can assume that d, c′>13>d′,c and that νc+(1−ν)c′=νd+(1−ν)d′=13. Consequently a + d ≥ 1 and b′ + c′ ≥ 1. Symmetry also allows us to assume that a + d′ ≥ 1. In particular, a>23; and, by the Fact, b′<23. So a′ + d ≥ 1, d>23, c′<23.

Now extend the lines that connect (a, b) to (a′,b′) and (c, d) to (c′,d′), by increasing a, b′, c′, d while decreasing a′, b, c, d′, until a′ = 1 – d and a = 1 – d′, and until either a = 1 or b = 0, and either d = 1 or c = 0. The only solution of this kind with b′ + c′ ≥ 1 occurs when a = d = 1, a′ = b = c = d′ = 0, b′ = c′ = 1/2, μ=13, ν=23.

(d) For the first solution, we can let W , X, Y, Z be uniform on {0, 1, 2}, {0, 1}, {0, 1, 2}, and {0}, respectively; and let A = {10, 20}, B = {11, 12}, C = {00}, D = {00}. (For example, WXY Z = 1110 gives event B.) The second solution turns out to be the same, but with (X, Y, Z, W ) in place of (W, X, Y, Z). Notice that the solution applies also to P_4, where the threshold is 13. [See STOC 43 (2011), 242.]

323. cbc. In this simple case, we just eliminate all strings in which c is followed by a.

324. For 1 ≤ j ≤ n, and for each v such that v = xj or v ––– xj, let i ≺ j for each i < j such that v = xi. (If several values of i qualify, it suffices to consider only the largest one. Several authors have used the term “dependence graph” for this partial ordering.) The traces equivalent to α correspond to the topological sortings with respect to ≺, because those arrangements of the letters are precisely the permutations that preserve the empilement.

In (136), for example, with x_1 ...xn = bcebafdc, we have 1 ≺ 2, 1 ≺ 4, 2 ≺ 4, 4 ≺ 5, 3 ≺ 6, 2 ≺ 7, 3 ≺ 7, 2 ≺ 8, 4 ≺ 8, and 7 ≺ 8. Algorithm 7.2.1.2V produces 105 solutions, 12345678 (bcebafdc) through 36127485 (efbcdbca).

325. Every such trace α yields an acyclic orientation, if we let u ––→ v when u appears at a lower level in α’s empilement. Conversely, the topological sortings of any acyclic orientation are all equivalent traces; so this correspondence is one-to-one. [See Ira M. Gessel, Discrete Mathematics 232 (2001), 119–130.]

326. True: x commutes with y if and only if y commutes with x.

327. Each trace α is represented by its height h = h(α) ≥ 0, and by h linked lists Lj = Lj(α) for 0 ≤ j < h. The elements of Lj are the letters on level j of α’s empilement; these letters have disjoint territories, and we keep each list in alphabetic order so that the representation is unique. The canonical string representing α is then L_0L_1 ...Lh–1. (For example, in (136) we have L_0 = be, L_1 = cf , L_2 = bd, L_3 = ac, and the canonical representation is becfbdac.) We also maintain the sets Uj = ∪{T (a) | a ∈ Lj} as bit vectors; in (136), for example, they are U_0 = #36, U_1 = #1b, U_2 = #3c, U_3 = #78.

To multiply α by β, do the following for k = 0, 1, ... , h(β) – 1 (in that order), and for each letter b ∈ Lk(β) (in any order): Set j ← h(α); then while j > 0 and T (b) & Uj–1(α) = 0, set j ← j – 1. If j = h(α), set Lj(α) empty, Uj(α) ← 0, and h(α) ← h(α) + 1. Insert b into Lj(α), and set Uj(α) ← Uj(α)+ T (b).

328. Do the following for k = h(β) – 1, ... , 1, 0 (in that order), and for each letter b ∈ Lk(β) (in any order): Set j ← h(α) – 1; while j > 0 and T (b)& Uj(α) = 0, set j ← j – 1. Report failure if b isn’t in Lj(α). Otherwise remove b from that list and set Uj(α) ← Uj(α) – T (b); if Uj(α) is now zero, set h(α) ← h(α) – 1.

If there was no failure, the resulting α is the answer.

329. Do the following for k = 0, 1, ... , h(α) – 1 (in that order), and for each letter a ∈ Lk(α) (in any order): Report failure if a isn’t in L_0(β). Otherwise remove a from that list, set U_0(β) ← U_0(β) – T (a), and renormalize the representation of β.

Renormalization involves the following steps: Set j ← c ← 1. While Uj–1(β) ≠ 0 and c ≠ 0, terminate if j = h(β); otherwise set c ← 0, j ← j+1, and then, for each letter b in Lj–1(β) such that T (b)& Uj–2(β) = 0, move b from Lj–1(β) to Lj–2(β) and set Uj–2(β) ← Uj–2(β)+ T (b), Uj–1(β) ← Uj–1(β) – T (b), c ← 1. Finally, if Uj–1(β) = 0, set Ui–1(β) ← Ui(β) and Li–1(β) ← Li(β) for j ≤ i < h(β), then set h(β) ← h(β) – 1.

If there was no failure, the resulting β is the answer.

330. Let the territorial universe be V ∪ E, the vertices plus edges of G, and let T (a) = {a} ∪ {{a, b} | a ––– b}. [G. X. Viennot, in 1985, called this subgraph a starfish.] Alternatively, we can get by with just two elements in each set T (a) if and only if G = L(H) is the line graph of some other multigraph H. Then each vertex a of G corresponds to an edge u ––– v in H, and we can let T (a) = {u, v}.

[Notes: The smallest graph G that isn’t a line graph is the “claw” K_1,3. Since sets of independent vertices in the line graph G are sets of disjoint edges in H, also called matchings of H, the Möbius polynomial of G is also known as the “matching polynomial” of H. Such polynomials are important in theoretical chemistry and physics. When all territories have |T (a)| ≤ 2, all roots of the polynomial MG∗(z) in (149) are real and positive, by exercise 341. But Mclaw(z, z, z, z) = 1 – 4z +3z^2 – z^3 has complex roots ≈ 0.317672 and 1.34116 ± 1.16154i.]

331. If α is a string with k > 0 occurrences of the substring ac, there are 2k ways to decompose α into factors {a, b, c, ac}, and the expansion includes +α and –α each exactly 2k–1 times. Thus we’re left with the sum of all strings that don’t contain ‘ac’.

332. No: If b commutes with a and c, but ac ≠ ca, we’re dealing with strings that contain no adjacent pairs ba or cb; hence cab qualifies, but it’s equivalent to the smaller string bca. [Certain graphs do define traces with the stated property, as we’ve seen in (135) and (136). Using the next exercise we can conclude that the property holds if and only if no three letters a < b < c have a―b, b―c, and a ––– c in the graph G of clashes. Thus the letters can be arranged into a suitable linear order if and only if G is a cocomparability graph; see Section 7.4.2.]

333. To show that Σα∈A,β∈B (–1)|β|αβ = 1, let γ = a_1 ...an be any nonempty string. If γ cannot be factored so that a_1 ...ak ∈ A and ak+1 ...an ∈ B, then γ doesn’t appear. Otherwise γ has exactly two such factorizations, one in which k has its smallest possible value and the other in which k is one greater; these factorizations cancel each other in the sum. [Discrete Mathematics 14 (1976), 215–239; Manuscripta Mathematica 19 (1976), 211–243. See also R. Fröberg, Mathematica Scandinavica 37 (1975), 29–39.]

334. Equivalently we want to generate all strings of length n on the alphabet {1,...,m} that satisfy the following criterion, which strengthens the adjacent-letter test of exercise 332: If 1 ≤ i < j ≤ n, xi―xj, xi+1―xj, ... , xj−1―xj, then xi ≤ xj. [See A. V. Anisimov and D. E. Knuth, Int. J. Comput. Inf. Sci. 8 (1979), 255–260.]

T1. [Initialize.] Set x_0 ← 0 and xk ← 1 for 1 ≤ k ≤ n.

T2. [Visit.] Visit the trace x_1 ...xn.

T3. [Find k.] Set k ← n. While xk = m set k ← k – 1. Terminate if k = 0.

T4. [Advance xk.] Set xk ← xk + 1 and j ← k – 1.

T5. [Is xk valid?] If xj > xk and xj―xk, return to T4. If j > 0 and xj < xk and xj―xk, set j ← j – 1 and repeat this step.

T6. [Reset xk+1 ...xn.] While k < n do the following: Set k ← k +1, xk ← 1; while xk–1 > xk and xk−1―xk, set xk ← xk + 1. Then go back to T2.

335. Given such an ordering, we have MG = det(I – A), where the entry in row u and column v of A is v[u ≥ v or u–––v]. The determinant in the given example is
det(1−b−c00001−b0−d000−b1−c−d−e00−b−c1−d0−f0−b−c−d1−e−f0−b−c−d−e1−f)+det (−a−b−c00001c−d00001−d−e00001−d0−f000−d1−e−f000−d−e1−f),
after expanding the first column, then subtracting the first row from all other rows in the right-hand determinant. Therefore this rule satisfies recurrence (142).

[The result also follows from MacMahon’s Master Theorem, exercise 5.1.2–20, using the characterization of lexicographically smallest traces in answer 334. According to Theorem 5.1.2B, such traces are in one-to-one correspondence with multiset permutations whose two-line representation does not contain vu when v > u and v―u. Is there a similar determinantal expression when G is not a cocomparability graph?]

336. (a) If α is a trace for G and β is a trace for H, we have μG⊕H(αβ) = μG(α)μH(β). Hence MG⊕H = MG MH. (b) In this case μG–––H(αβ) = μG(α) if β = ∊, μH(β) if α = ∊; otherwise it’s zero. Therefore MG–––H = MG + MH – 1.

[These rules determine MG recursively whenever G is a cograph (see exercise 7–90). In particular, complete bipartite and k-partite graphs have simple Möbius series, exemplified by MG = (1 – a)(1 – b)(1 – c) + (1 – d)(1 – e)+(1 – f) – 2 when G = K_3,2,1.]

337. Substituting a_1 + … + ak for a in MG gives MG′. (Each trace for G′ is obtained by putting subscripts on the a’s of the traces for G.)

338. The proof of Theorem F needs only minor changes: We limit α to traces that contain no elements of A, and we define α′ and β′ by letting a be the smallest letter ∉ A in the bottom level of γ’s empilement. If γ has no such letter, there’s only one factorization, with α = . Otherwise we pair up cancelling factorizations. [Incidentally, the sum of all traces whose sinks are in A must be written in the other order: MG−1⁢ MG\A.]

339. (a) “Push down” on piece xj and factor out what comes through the floor.

(b) Factor out the pyramid for the smallest label, and repeat on what’s left.

(c) This is a general convolution principle for labeled objects [see E. A. Bender and J. R. Goldman, Indiana Univ. Math. J. 20 (1971), 753–765]. For example, when l = 3 the number of ways to get a labeled trace of length n from three labeled pyramids is Σi,j,k   (ni,j,k)PiPjPk/3!=n!  Σi,j,k(Pi/i!)(Pj/j!)(Pk/k!)/3!, with i+j +k = n in both of these sums. We divide by 3! so that the topmost pyramid labels will be increasing.

(d) Sum the identity in (c) for l = 0, 1, 2, ....

(e) T(z) = Σn≥0 tnzn = 1/MG(z) by Theorem F, and P(z) = Σn≥1 pnzn/n. Note: If we retain the letter names, writing for example MG(z) = 1–(a+b+c)z +acz^2 instead of MG(z) = 1–3z+z^2, the formal power series – ln MG(z) exhibits the pyramids of length n in the coefficient of zn, but only in the sense of commutative algebra (not trace algebra). For example, the coefficient of z^3 obtained from Σk≥1(1 – MG(z))k/k with trace algebra includes the nonpyramidal term bac/6.

340. Let w((i_1 ... ik)) = (–1)k–1ai_1i_2ai_2i_3 ...aiki_1; thus w(π) = (–a_13a_34a_42a_21) (–a_57a_75)(a_66) in the given example. The permutation polynomial is then det A, by definition of the determinant. (And we get the permanent, if we omit the (–1)k–1.)

341. The hint is true when n = 2, since the first involution polynomials are w_11x and w_11w_22x^2 – w_12. And there’s a recurrence: W(S) = wiixW(S \ i) – Σj≠i W(S \{i, j}).

So we can prove the existence of n + 1 roots s_1 < r_1 < … < rn < sn+1 by induction: Let Wn(x) be the polynomial for {1,...,n}. Then Wn+1(x) is w(n+1)(n+1)xWn(x) minus n polynomials w(n+1)jW ({1,...,n} \ j), each with roots qk(j) that are nicely sandwiched between the roots of Wn. Furthermore qn−k(j)=−qk(j) and rn+1–k = –rk, for 1 ≤ k ≤ n/2. It follows that Wn+1(rn) < 0, Wn+1(rn–1) > 0, and so on, with (–1)kWn+1(rn+1–k) > 0 for 1 ≤ k ≤ n/2. Moreover, Wn+1(0) = 0 when n is even; (–1)kWn+1(0) > 0 when n = 2k – 1; and Wn+1(x) > 0 for all large x. Hence the desired sk exist. [See Heilmann and Lieb, Physical Review Letters 24 (1970), 1412.]

342. If we replace (i_1 ... ik) by ai_1i_2 ai_2i_3 ... aiki_1 (as in answer 340, but without the (–1)k–1), then MGn becomes det(I – A). Replacing aij by aijxj gives the determinant in MacMahon’s Master Theorem. And if x_1 = … = xn = x, we get the polynomial det(I–xA), whose roots are the reciprocals of the roots of A’s characteristic polynomial.

343. The formulas in answer 336 show that MG(p_1,...,pm) increases whenever any pj decreases, with respect to a cograph G. The only graph on ≤ 4 vertices that isn’t a cograph is P_4 (see exercise 7–90); then MG(p_1,p_2,p_3,p_4) = 1 – p_1 – p_2 – p_3 – p_4 + p_1p_3 + p_1p_4 + p_2p_4 = (1 – p_1)(1 – p_3 – p_4) – p_2(1 – p_4). In this case also we can conclude that MG(p_1,...,p_4) > 0 implies (p_1,...,p_4) ∈ R(G). But when G = P_5, we find MG(1 – ∊, 1 – ∊, ∊, 1 – ∊, 1 – ∊) > 0 for 0 ≤ ∊ < ϕ–2; yet (1 – ∊, 1 – ∊, ∊, 1 – ∊, 1 – ∊) is never in R(G) because MG(0, 0, ∊, 1 – ∊, 1 – ∊) = –(1 – ∊)2.

344.(a) If some minterm, say B1B¯2B¯3B4, has negative “probability,” then p_1p_4 × (1 – π_2 – π_3 + π_23) < 0; hence MG(0,p_2,p_3, 0) < 0 violates the definition of R(G).

(b) In fact, more is true: πI∪J = πIπJ when i―j j for i ∈ I, j ∈ J, and I ∩ J = ∅.

(c) It’s MG(p_1[1 ∈ J ],...,pm[m ∈ J ]), by (140) and (141). This important fact, already implicit in the solution to part (a), implies that β(G | J) > 0 for all J.

(d) Writing just ‘J’ for ‘G|J’, we shall prove that α(i ∪ J)/β(i ∪ J) ≥ α(J)/β(J) for i ∉ J, by induction on |J|. Let J′={j∈J⁢ |  i―j}. Then we have
α(i∪J)=α(J)−Pr(Ai∩∩j∈JA¯j)≥α(J)−Pr(Ai∩∩j∈J′A¯j)≥α(J)−piα(J′),
because of (133). Also β(i ∪ J) = β(J) – piβ(J′). Hence α(i ∪ J)β(J) – α(J)β(i ∪ J) ≥ (α(J) – piα(J′))β(J) – α(J)(β(J) – piβ(J′)) = pi(α(J)β(J′) – α(J′)β(J)), which is ≥ 0 by induction since J′ ⊆ J.

[This argument proves that Lemma L holds whenever (p_1,...,pm) leads to a legitimate probability distribution with β(G) > 0; hence such probabilities are in R(G).]

(e) By induction, we have β(i ∪ J) = β(J) – θiβ(J′) Πi–––j (1 – θj) ≥ β(J) – θiβ(J′) Πj∈J\J′ (1 – θj) ≥ (1 – θi)β(J), because β(J)/β(J′) ≥ Πj∈J\J′ (1 – θj).

345. (Solution by A. D. Scott and A. D. Sokal.) Set pj′=(1+δ)pj where δ ≤ 0 is the slack of (p_1,...,pm). Then MG(p1′,…,pm′)=0, but it becomes positive if any p′j is decreased. Define events B1′,+,Bm′ by the construction in exercise 344. Let C_1, ... , Cm be independent binary random variables such that Pr(Cj = 1) = qj, where (1−pj′)(1−qj)=1−pj. Then the events Bj=Bj′∨Cj satisfy the desired conditions: Pr(Br  |  B¯ji ∩…∩ B¯jk)=Pr(Bi  | B¯ji′ ∩…∩ B¯jk′)=Pr(Bi)=pi; and Pr(B1⁢ ∨…∨Bm)≥Pr(B1′∨…∨⁢Bm′ )=1.

346. (a) By (144), Ka,G is the sum of all traces on the probabilities of G \ a whose sources are neighbors of a. Decreasing pj doesn’t decrease any trace.

(b) Suppose vertex a = 1 has neighbors 2, ... , j. If we’ve recursively computed MG\a∗ and MG\a, finding that (pj+1, ... , pm) ∈ R(G\a∗) and (p_2, ... , pm) ∈ R(G\a), then we know Ka,G; and the monotonicity property in (a) implies that (p_1, ... , pm) ∈ R(G) if and only if aKa,G < 1.

The graph in exercise 335 can, for example, be processed as follows:

Mabcdef=Mbcdef(1−aMdefMbcdef)=(1−a′)(1−b′)…(1−f′),a′=a(1−b′)(1−c′),Mbcdef=Mcdef(1−bMcefMcdef)=(1−b′)(1−c′)…(1−f′),b′=b(1−c″)(1−c′)(1−d′),Mcdef=Mdef(1−cMfMdef)=(1−c′)(1−d′)(1−e′)(1−f′),c′=c(1−d′)(1−e′),Mcef=Mef(1−cMfMef)=(1−c″)(1−e′)(1−f′),c″=c(1−e′),Mdef=Mef(1−dMeMf)=(1−d′)(1−e′)(1−f′),d′=d(1−e″)(1−e′)(1−f′),Mef=Mf(1−eM∊Mf)=(1−e′)(1−f′),e′=e(1−f′),Me=M∊(1−eM∊M∊)=(1−e″),e″=e,Mf=M∊(1−fM∊M∊)=(1−f′),f′=f,
with M∊ = 1. (The equations on the left are derived top-down, then the equations on the right are evaluated bottom-up. We have (a,b,...,f)∈ R(G) if and only if f′ < 1, e′ < 1, e′ < 1, ... , a′ < 1.) Even better is to traverse this graph in another order, using the rule MG⊕H = MG MH (exercise 336) when subgraphs aren’t connected:

Mcdabef=Mdabef(1−cMbMfMdabef)=(1−c′)(1−d1)+(1−f′),  c′=c(1−a′)(1−d′)(1−e′),Mdabef=MabMef(1−dMaMeMabMef)=(1−d′)(1−a′)(1−b′)(1−e′)(1−f′),  (see⁢  below)

Mab=Mb(1−aMϵMb)=(1−a′)(1−b′),a′=a(1−b′),Ma=Mϵ(1−aMϵMϵ)=(1−a″),a″=a,Mb=Mϵ(1−b(MϵMϵ)=(1−b′),b′=b,
where d′ = dMaM∊/(MabMef ) = d(1 – a″)(1 – e″)/((1 – a′)(1 – b′)(1 – e′)(1 – f′)), and Mef , Me, Mf , M are as before. In this way we can often solve the problem in linear time. [See A. D. Scott and A. D. Sokal, J. Stat. Phys. 118 (2005), 1151–1261, §3.4.]

347. (a) Suppose v_1 ––– v_2 –––…––– vk ––– v_1 is an induced cycle. We can assume that v_1 ≻ v_2. Then, by induction on j, we must have v_1 ≻ … ≻ vj for 1 < j ≤ k; for if vj+1 ≻ vj we would have vj+1 ––– vj–1 by (∗). But now vk ––– v_1 implies that k = 3.

(b) Let the vertices be {1,...,m}, with territory sets T (a) ⊆ U for 1 ≤ a ≤ m; and let U be a tree such that each U | T (a) is connected. Let Ua be the least common ancestor of T (a) in U. (Thus the nodes of T (a) appear at the top of the subtree rooted at Ua.) Since Ua ∈ T (a), we have a ––– b when Ua = Ub.

Writing s ≻ U t for the ancestor relation in U, we now define a ≻ b if Ua ≻U Ub or if Ua = Ub and a < b. Then (∗) is satisfied: If t ∈ T (a) ∩ T (b), we have Ua ≻U t and Ub ≻U t, hence Ua≻¯U  Ub or Ub≻¯U  Ua, hence a ≻ b or b ≻ a. And if a ≻ b ≻ c and t ∈ T (a) ∩ T (c), we have Ua≻¯U  Ub≻¯U   Uc; consequently Ub ∈ T (a) ∩ T (b), because Ub lies on the unique path between t and Ua in U and T (a) is connected.

(c) Processing the nodes in any order such that a is eliminated before b whenever Ua is a proper ancestor of Ub will then lead only to subproblems in which the algorithm needs no “double-primed” variables.

For example, using (a,b,...,g) instead of (1, 2,..., 7) in order to match the notation in exercise 346, suppose U is the tree rooted at p having the edges p ––– q, p ––– r, r ––– s, r ––– t, and let T (a) = {p, q, r, t}, T (b) = {p, r, s}, T (c) = {p, q}, T (d) = {q}, T (e) = {r, s}, T (f) = {s}, T (g) = {t}. Then a ≻ b ≻ c ≻ d, c ≻ e ≻ f, e ≻ g. The algorithm computes Mabcdefg = (1 – a′)Mbcdefg, Mbcdefg = (1 – b′)Mcdefg, etc., where a′ = aMf /Mbcdefg, b′ = bMdfg/Mcdefg = b(MdMf Mg)/(McdMef Mg), etc.

In general, the tree ordering guarantees that no “double-primed” variables are needed. Thus the formulas reduce to v′ = v/Πu––v, v≻u (1 – u′) for each vertex v.

(d) For example, we have p_1 = a, ... , p_7 = g, θ_1 = a′, ... , θ_7 = g′ in (c). The values of the θ’s, which depend on the ordering ≻, are uniquely defined by the given equations; and we have MG(p_1, ... , pm) = (1 – θ_1) ... (1 – θm) in any case. [W. Pegden, Random Structures & Algorithms 41 (2012), 546–556.]

348. There is at least one singularity at z = ρeiθ for some θ. If 0 < r < ρ, the power series f(z)=Σn=0∞ f(n) (reiθ)(z−reiθ)n/n! has radius of convergence ρ–r. If z = ρ isn’t a singularity, the radius of convergence for θ = 0 would exceed ρ – r. But |f(n)(reiθ)|=|Σm=0∞ mn¯an(reiθ)m−n|≤f(n)(r). [Mathematische Annalen 44 (1894), 41–42.]

349. Typical generating functions are g0000001 = 1; g0110110 = z(g0100110 + g0101110 + g0110110+g0111110)/4 (in Case 1) or g0110110 = z(g0000110+g0010110+g0100110+g0110110)/4 (in Case 2). These systems of 128 linear equations have solutions whose denominators involve one or more of the polynomials 4–z,2–z,16–12z+z^2,4–3z,64–80z+24z^2–z^3, 8 – 8z + z^2 in Case 1 (see exercise 320); the denominators in Case 2 are powers of 4 – z.

Setting g(z) = Σx gx(z)/128 leads to g(z) = 1/((2 – z)(8 – 8z + z^2)) in Case 1, with mean 7 and variance 42; g(z) = (1088 – 400z +42z^2 – z^3)/(4 – z)6 in Case 2, with mean 1139/729 ≈ 1.56 and variance 1139726/729^2 ≈ 2.14.

[The upper bound E_1 + … + E_6 is achieved by the distribution of Case 1, because it matches the extreme distribution (148) of the path graph P_6. Incidentally, if Case 1 is generalized from n = 7 to arbitrary n, the mean is n(n – 1)/6 and the variance is (n + 3)(n +2)n(n – 1)/90.]

350. (a) The generating function for N is Πk=1n(1−ξk)/(1−ξkz); so the mean and variance, in general, are Σk=1n ξk/(1−ξk) and Σk=1n ξk/(1−ξk)2. In particular, the means are (i) n; (ii) n/(2n – 1); (iii) n/(2n – 1); (iv) H2n⁢ −Hn+12n=ln 2+O(1/n); (v) 12(1n+1+12n−12n+1)=12n+O(1/n2). The variance in case (i) is 2n; otherwise it’s asymptotically the same as the mean, times 1 + O(1/n).

(b) In this case the mean and variance are ξ/(1 – ξ) and ξ/(1 – ξ)2, where ξ = Pr(Am) = 1 – (1 – ξ_1) ... (1 – ξn). This value ξ is (i) 1 – 2–n; (ii) 1−(1−12n)n=1−e−1/2+O(1/n); (iii) 1 – (1 – 2–n)n = n/2n + O(n^2/4n); (iv) 1/2; (v) 1/(2n + 2). Hence the respective means are (i) 2n – 1; (ii) e1/2 – 1+ O(1/n); (iii) n/2n + O(n^2/4n); (iv) 1; (v) 1/(2n + 1). And the variances are (i) 4n – 2n; (ii) e – e1/2 + O(1/n); (iii) n/2n + O(n^2/4n); (iv) 2; (v) 1/(2n +1)+1/(2n +1)2.

(c) Since G is Kn,1, exercises 336 and 343 imply that (ξ_1,...,ξn,ξ) ∈ R(G) if and only if ξ<12. This condition holds in cases (ii), (iii), and (v).

351. (Solution by Moser and Tardos.) We require i ––– j if there’s a setting of the variables such that Ai is false and Aj is true, provided that some change to the variables of Ξj might make Ai true. And vice versa with i ↔ j.

(The Local Lemma can be proved also for directed lopsidependency graphs; see Noga Alon and Joel H. Spencer, The Probabilistic Method (2008), §5.1. But the theory of traces, which we use to analyze Algorithm M, is based on undirected graphs, and no algorithmic extension to the directed case is presently known.)

352. Answer 344(e), with MG = β(i∪J), MG\i = β(J), proves that MG\i/MG ≥ 1–θi.

353. (a) There are n + 1 sorted strings in Case 1, namely 0k1n–k for 0 ≤ k ≤ n. There are Fn+2 solutions in Case 2 (see, for example, exercise 7.2.1.1–91).

(b) At least 2nMG(1/4), where G is the path Pn–1. By exercise 320 we have MG(1/4) = fn–1(1/4) = (n +1)/2n; so Case 1 matches the lower bound.

(c) There are no lopsidependencies. Hence the relevant G is the empty graph on m = n–1 vertices; MG(1/4) = (3/4)n–1 by exercise 336; and indeed, Fn+2 ≥ 3n–122–n.

354. Differentiate (151) and set z ← 1.

355. If A = Aj is an isolated vertex of G, then 1 – pjz is a factor of the polynomial MG∗(z) in (149), hence 1 + δ ≤ 1/pj; and Ej = pj/(1 – pj) ≤ 1/δ. Otherwise MG(p1,…,pj−1,pj(1+δ),pj+1,…,pm)=MG∗(1)−δpjMG/A∗∗(1)>MG∗(1+δ)=0; so Ej=pjMG/A∗∗(1)/MG∗(1)>1/δ.

356. (a) We prove the hint by induction on |S|. It’s obvious when S = ∅; otherwise let X = S ∩ ∪i∈Uj Uj and Y = S \ X. We have

Pr(Ai|A¯S)=Pr(Ai∩A¯X∩A¯Y)Pr(A¯X∩A¯Y)≤Pr(Ai∩A¯Y)Pr(A¯X∩A¯Y)≤Pr(Ai)    PrA¯Y)Pr(A¯X∩A¯Y)=Pr(Ai)Pr(A¯X|A¯Y)

by (133). Suppose i belongs to the cliques Uj_0, ... , Ujr where j = j_0. Let X_0 = ∅ and Xk = (S ∩ Ujk) \ Xk–1, Yk = Y ∪ X_1 ∪ … ∪ Xk–1 for 1 ≤ k ≤ r. We have Pr(Al | ĀYk) ≤ θljk for all l ∈ Xk, since |Yk| < |S| when Xk ≠ = ∅; hence Pr(ĀXk | ĀYk) ≥ (1 + θijk – Σjk). Thus Pr(ĀX |ĀY) = Pr(ĀX_1 |ĀY_1) Pr(ĀX_2 |ĀY_2) ... Pr(ĀXr |ĀYr) ≥ Πk≠j,i∈Uk (1 + θik – Σk), by the chain rule (exercise MPR-14); the hint follows.

Finally let Wk = U_1 ∪…∪ Uk for 1 ≤ k ≤ t. The hint implies that

Pr(A¯1∩…∩A¯m)=Pr(A¯W1)  Pr(A¯W2|A¯W1)…  Pr(A¯Wt|A¯Wt−1)≥(1−Σ1)(1−Σ2)…(1−Σt)>0.

(b) The extreme events B_1, ... , Bm of Theorem S satisfy the hint of (a). Thus Pr(Bi|∩k∉Uj  B¯k)≤θij for all i ∈ Uj; hence qi=Pr(Bi|∩k≠i  B¯k)≤θij/(1+θij−Σj). Furthermore Ei = qi/(1 – qi) in (152), because qi = piMG\i∗/MG\i.

(c) Let U_1, ... , Ut be the edges of G, with θik = θi when Uk = {i, j}. Then Σk = θi +θj < 1, and the sufficient condition in (a) is that Pr(Ai) ≤ θi Πj≠k,i–––j(1–θj) whenever i ––– k. (But notice that Theorem M does not hold for such larger pi.)

[K. Kolipaka, M. Szegedy, and Y. Xu, LNCS 7408 (2012), 603–614.]

357. If r > 0, we have x = r/(1–p), y = r/(1–q). But r = 0 is possible only on the axes of Fig. 94: Either (p, q) = (0, 1), x = 0, 0 < y ≤ 1, or (p, q) = (1, 0), 0 < x ≤ 1, y = 1.

358. Suppose x ≥ y (hence p ≥ q and X > 0). Then p ≤ r if and only if 1 – y ≤ y.

359. Instead of computing πl by formula (154), represent it as two numbers (πl+,πl′), where πl+ is the product of the nonzero factors and πl′ is the number of zero factors. Then the quantity πl¯ needed in (156) is πl¯+[πl¯′=0]; and the quantity πl/(1 – ηC→l) is πl+[πl′=1] if ηC→l = 1, otherwise it’s πl+[πl′=0]/(1−ηC→l). A similar method can be used to separate out the zero factors of Πl∈C γl→C in (157).

360. We may assume that η_3 = 0. Since πl = 1 implies that ηC→l=γl¯→C=0, we have ηC→1=ηC→2¯=ηC→3=ηC→4¯=γ1¯→C=γ2→C=γ3¯→C=γ4→C=0 for all C. Consequently, as in (159), all but three of the values ηC→l are zero; let x, y, z denote the others. Also let η1¯=a,η2=b,η4=c,η3¯=d. Then π1¯=(1−a)(1−x), π_2 = (1–b)(1–y), π_4 = (1–c)(1–z), and π3=1−d. A fixed point is obtained if x = d(b + cd(1–b)+ ad^2(1–b)(1–c))/(1 – d^3(1–a)(1–b)(1–c)), etc. If d is 0 or 1 then x = y = z = d. [Are there any other fixed points, say with π_1 ≠ 1?]

361. The π’s and γ’s will also be either 0 or 1, and we exclude the case πl=πl¯=0; thus each variable v is either 1, 0, or ∗, depending on whether (πv,πv¯)is(0, 1), (1, 0), or (1, 1).

Any assignment of 1, 0, or ∗ to the variables is permissible, provided that every clause has at least one literal that’s true or two that are ∗. (Such partial assignments are called “covering,” and they’re usually possible even with unsatisfiable clauses; see exercise 364.) All survey messages ηC→l′=ηC→l are zero except when clause C has l as its only non-false literal. The reinforcement message ηl can be either 0 or 1, except that it must be 1 if l is true (πl = 0) and all messages ηC→l are 0.

If we also want ηl′=ηl, we take κ = 1 in (158), and ηl = 1 – πl.

362. Create a linked list L, containing all literals that are to be forced true, including all literals that are in 1-clauses of the original problem. Do the following steps while L is nonempty: Remove a literal l from L; remove all clauses that contain l; and remove l¯ from all the clauses that remain. If any of those clauses has thereby been reduced to a single literal, (l′), check to see if l′ or l¯′ is already present in L. If l¯′ is present, a contradiction has arisen; we must either terminate unsuccessfully or restart step S8 with increased ψ. But if l¯′ and l′ are both absent, put l′ into L.

363. (a) True; indeed, this is an important invariant property of Algorithm C.

(b) W (001) = 1, W (∗∗∗) = p_1p_2p_3, otherwise W (x) = 0.

(c) Statements (i) and (iii) are true, but not (ii); consider x = 10∗, x′ = 00∗, and the clause 123.

(d) All eight subsets of {1,2¯,3¯} are stable except {2¯,3¯}, because x_1 is constrained in 100. The other seven are partially ordered as shown. (This diagram illustrates L_7, the smallest lattice that is lower semimodular but not modular.)

(e)

x_2x_3 = 00

01

0∗

10

11

1∗

∗0

∗1

∗∗

x_1 = 0

0

q_1q_2

0

q_1q_3

q_1q_2q_3

q_1q_2p_3

0

q_1p_2q_3

q_1p_2p_3

x_1 = 1

q_2q_3

q_1q_2q_3

q_1q_2p_3

q_1q_2q_3

q_1q_2q_3

q_1q_2p_3

q_1p_2q_3

q_1p_2q_3

q_1p_2p_3

x_1 = ∗

0

p_1q_2q_3

p_1q_2p_3

p_1q_2q_3

p_1q_2q_3

p_1q_2p_3

p_1p_2q_3

p_1p_2q_3

p_1p_2p_3

(f) One solution is {1¯2¯34¯5¯,1¯4,2¯5,3¯45¯,3¯4¯5}. (For these clauses the partial assignment {3} is stable, but it is “unreachable” below {1, 2, 3, 4, 5}.)

(g) If L = L′ \ l and L′ ∈ ℒ but L /∉ ℒ, introduce the clause (x1∨∨k∈L′x¯k).

(h) True, because L′ = L \ l′ and L″ = L \ l″, where |l′| and |l″| are unconstrained with respect to L. A variable that’s unconstrained with respect to L is also unconstrained with respect to any subset of L.

(i) Suppose L′ = L′(0) ≺ … ≺ L′(s) = {1,...,n} and L″ = L″(0) ≺ … ≺ L″(t) = {1,...,n}. Then L′(s–i) ∩ L″(t–j) is stable for 0 ≤ i ≤ s and 0 ≤ j ≤ t, by induction on i + j using (h).

(j) It suffices to consider the case L = {1,...,n}. Suppose the unconstrained variables are x_1, x_2, x_3. Then, by induction, the sum is q_1q_2q_3 + p_1 + p_2 + p_3 – (p_1p_2 + p_1p_3 + p_2p_3)+ p_1p_2p_3 = 1, using “inclusion and exclusion” to compensate for terms that are counted more than once. A similar argument works with any number of unconstrained variables.

Notes: See F. Ardila and E. Maneva, Discrete Mathematics 309 (2009), 3083– 3091. The sum in (j) is ≤ 1 when each pk+qk ≤ 1 for 1 ≤ k ≤ n, because it is monotone. Because of (i), the stable sets below L form a lower semimodular lattice, with

L′∧L″=L′∩L″andL′∨L″=∩{L′′′|L′′′⊇ L′∪L″  and  L′′′⊑L}.

E. Maneva and A. Sinclair noted in Theoretical Comp. Sci. 407 (2008), 359–369 that a random satisfiability problem is satisfiable with probability ≤ E W (X), the expected total weight of partial assignments having the given distribution, because of identity (j); this led them to sharper bounds than had previously been known.

364. (a) True if and only if all clauses have length 2 or more.

(b) 001 and ∗∗∗ are covering; these are the partial assignments of nonzero weight, when q_1 = … = qn = 0 in the previous exercise. Only 001 is a core.

(c) ∗∗∗ is the only covering and the only core; W (0101) = W (0111) = q_3.

(d) In fact, every stable partial assignment L′ has a unique covering assignment L with L ⊑ L′, namely L = ∩{L″ | L″ ⊑ L′, obtained by successively removing unconstrained literals (in any order)}.

(e) If L′ and L″ are adjacent we have L′ ∩ L″ ⊑ L′ and L′ ∩ L″ ⊑ L″.

(f) Not necessarily. For example, the clauses {1¯2¯34,1¯23¯4,1¯23¯4,12¯3¯4,12¯34¯,123¯4¯} define S¯2(x1,x2,x3,x4); there are two clusters but only an empty core.

[A. Braunstein and R. Zecchina introduced the notion of covering assignments in J. Statistical Mechanics (June 2004), P06007:1–18.]

365. If L is any of the six solutions in (8), and if q is odd, then qL–d is a covering assignment for 0 ≤ d < q and 8q–d ≤ n < 9q–d. (For example, if L={1¯,2¯,3,4,5¯,6¯,7,8} the partial assignment 3L−1={2¯,5¯,8,11,14¯,17¯,20,23} works for n ∈ [23 .. 25].) Thus all n > 63 are “covered.” [Do all nonempty coverings of waerden (3, 3; n) have this form?]

366. Eliminating variable 1 (x_1) by resolution yields the erp rule x¯1←(x2∨x¯3)∧(x3∨x4), and new clauses {23¯4,23¯4¯,234,2¯34}. Then eliminating 2 (x_2) yields x_2 ← (x_3 ∨ x_4) ∧ (x¯3∨x4) and new clauses {34,3¯4}. Now 4 (x_4) is pure; so x_4 ← 1, and F′ = ∅ is satisfiable. (Going backwards in the erp rules will then make x_4 ← 1, x_2 ← 1, x_1 ← 0, regardless of x_3.)

367. (We can choose whichever of the two assignments is most convenient, for example by picking the shortest, since either one is a valid erp rule.) Any solution will either satisfy all the clauses on the right side of x¯ or all the clauses on the right side of x, or both. For if a solution falsifies both Ci \ x and Ci′\x¯, it falsifies Ci⋄Ci′.

In either case the value of x will satisfy all of the clauses C_1, ..., Ca, C1′,…,Cb′.

368. If (l) is a clause, subsumption removes all other clauses that contain l. Then resolution (with p = 1) will remove l¯ from all q of its clauses, and (l) itself.

369. Let Ci = (l ∨αi) and Cj′=(l¯∨βj). Each omitted clause Ci⋄Cj′=(αi∨βj), where 1 < i ≤ p and r < j ≤ q, is redundant, because it is a consequence of the non-omitted clauses (αi∨l¯1),…,(αi∨l¯r), (l_1 ∨ … ∨ lr ∨ βj) via hyperresolution. [This technique is called “substitution,” because we essentially replace |l| by its definition.]

370. (a∨b)∧(a∨b¯∨c¯)∧(a¯∨b∨c)=(a∨c¯)∧(b∨c). (See the discussion following 7.1.1–(27). In general, advanced preprocessors use the theory of DNF minimization, in its dual form, to find irredundant minimum forms for CNF. Such techniques are not implemented, however, in the examples of preprocessing considered in this section.)

371. One scenario starts by eliminating variable 1, replacing eight clauses by eight new ones: 234¯7¯,2¯3¯47,235¯9¯,2¯3¯59,34¯57¯,3¯45¯7,45¯79¯,4¯57¯9. Then 8 is eliminated, replacing another eight by eight: 24¯56¯,2¯45¯6,256¯7¯,2¯5¯67,257¯9¯,2¯5¯79,467¯9¯,4¯6¯79. Then come self-subsumptions: 234¯7¯↦237¯ (via 234), 34¯57¯↦357¯ (345), 357 ↦ 35 (357¯); and 35 subsumes 345, 357¯. Further self-subsumptions yield 235¯9¯↦239¯,2¯3¯59↦2¯3¯9, 2¯5¯79↦2¯79,2¯45¯6↦2¯46, 246 ↦ 46; and 46 subsumes 456, 467¯9¯,2¯46. Similarly, 2¯5¯67↦2¯67,4¯57¯9↦4¯59,2¯3¯47↦2¯3¯7,3¯45¯7↦3¯5¯7,3¯5¯7¯↦3¯5¯; and 3¯5¯ subsumes 3¯4¯5¯, 3¯5¯7. Then 24¯56¯↦24¯6¯,2¯4¯6¯↦4¯6¯; and 4¯6¯ subsumes 4¯5¯6¯,24¯6¯,4¯6¯79. Also 256¯7¯↦26¯7¯, 45¯79¯↦45¯9¯,257¯9¯↦27¯9¯.

Round 2 of variable elimination first gets rid of 4, replacing six clauses by just four using exercise 369: 236¯,2¯3¯6,569,5¯6¯9¯. Then variable 3 goes away; ten clauses become eight, again via exercise 369: 25¯6¯,2¯56,25¯7¯,2¯57,25¯9¯,2¯59,56¯9¯,5¯69. And the ten clauses that now contain 2 or 2¯ resolve into just four: 567¯9¯,56¯7¯9,5¯679¯,5¯6¯79.

After eliminating 7 and 9, only four clauses remain, namely 56, 56¯,5¯6,5¯6¯; and they quickly produce a contradiction.

372. (This problem is surprisingly difficult.) Are the clauses {1¯5¯,1¯6¯,2¯5¯,2¯6¯,3¯7¯,3¯8¯, 4¯7¯,4¯8¯, 123, 124, 134, 234, 567, 568, 578, 678} as “small” as possible?

373. Using the notation of (102), elimination of x_1m, x_2m, ... , xmm produces new clauses Mimk′ for 1 ≤ i, k < m as well as Mm(m–1). Then elimination of xm(m–1) gives (Mi(m–1) ∨ Mm(m–2)) for 1 ≤ i < m. This clause self-subsumes to Mi(m–1), usingMim1′,…Mim(m−2)′ . And Mi(m–1) subsumes each Mimk′, so we’ve reduced m to m–1. 374. As in (57), variables are numbered 1 to n, and literals from 2 to 2n + 1. But we will now number the clauses from 2n + 2 to m + 2n + 1. The literals of clauses will be stored in cells, somewhat as in Algorithm A, but with additional links: Each cell p contains not only a literal L(p), a clause number C(p), and forward/backward pointers F(p) and B(p) to other cells with the same literal, but also left/right pointers S(p) and D(p) to other cells in the same clause. (Think “sinister” and “dexter.”) Cells 0 and 1 are reserved for special use; cell l, for 2 ≤ l < 2n + 2, serves as the head of the doubly linked list of cells that contain the literal l; cell c, for 2n +2 ≤ c < m +2n + 2, serves as the head of the doubly linked list of cells that contain the elements of clause c; and cell p, for m +2n +2 ≤ p < M, either is available for future use or holds literal and clause data for a currently active clause.

Free cells are accessed via a global pointer AVAIL. To get a new p ⇐ AVAIL when AVAIL ≠ 0, we set p ← AVAIL, AVAIL ← S(AVAIL); but if AVAIL = 0, we set p ← M and M ← M + 1 (assuming that M never gets too large). To free one or more cells from p′to p″ that are linked together via left links, we set S(p′) ← AVAIL and AVAIL ← p″.

The number of active clauses containing literal l, TALLY(l), can therefore be computed as follows: Set t ← 0, p ← F(l); while not lit (p), set t ← t + 1 and p ← F(p); set TALLY(l) ← t; here ‘lit (p)’ stands for ‘p < 2n+2’. The number of literals in clause c, SIZE(c), can be computed by a similar loop, using ‘cls (p)’ to stand for ‘p < m+2n+2’: Set t ← 0, p ← S(c); while not cls (p), set t ← t + 1 and p ← S(p); set SIZE(c) ← t. After initialization, the TALLY and SIZE statistics can be updated dynamically as local changes are made. (TALLY(l) and SIZE(c) can be maintained in L(l) and C(c).)

To facilitate resolution, the literals of each clause are required to increase from left to right; in other words, we must have L(p) < L(q) whenever p = S(q) and q = D(p), unless cls (p) or cls (q). But the clauses within literal lists need not appear in any particular order. We might even have C(F(p)) > C(q) but C(F(p′)) < C(q′), when C(p) = C(p′) and C(q) = C(q′).

To facilitate subsumption, each literal l is assigned a 64-bit signature SIG(l) = (1 ≪ U_1) | (1 ≪ U_2), where U_1 and U_2 are independently random 6-bit numbers. Then each clause c is assigned a signature that is the bitwise OR of the signatures of its literals: Set t ← 0, p ← S(c); while not cls (p), set t ← t | SIG(L(p)) and p ← S(p); set SIG(c) ← t. (See the discussion of Bloom’s superimposed coding in Section 6.5.)

(a) To resolve c with c′, where c contains l and c′ contains l¯, we essentially want to do a list merge. Set p ← 1, q ← S(c), u ← L(q), q′ ← S(c′), u′ ← L(q′), and do the following while u + u′ > 0: If u = u′, copy(u) and bump(q, q′); if u = ū′ = l, bump(q, q′); if u = ū′ ≠ l, terminate unsuccessfully; otherwise if u > u′, copy(u) and bump(q); otherwise copy(u′) and bump(q′). Here ‘copy(u)’ means ‘set p′ ← p, p ⇐ AVAIL, S(p′) ← p, L(p) ← u’; ‘bump(q)’ means ‘set q ← S(q); if cls (q) set u ← 0, otherwise set u ← L(q)’; ‘bump(q′)’ is similar, but it uses q′ and u′; and ‘bump(q, q′)’ means ‘bump(q) and bump(q′)’. Unsuccessful termination occurs when clauses c and c′ resolve to a tautology; we set p ← 0, after first returning cells p through S(1) to free storage if p ≠ 1. Successful termination with u = u′ = 0 means that the resolved clause consists of the literals in cells from p through S(1), linked only via S pointers.

(b) Find a literal l in C with minimum TALLY(l). Set p ← F(l), and do the following while not lit (p): Set c′ ← C(p); if c′ ≠ c and ∼SIG(c′) & SIG(c) = 0 and SIZE(c′) ≥ SIZE(c), do a detailed subsumption test; then set p ← F(p). The detailed test begins with q ← S(c), u ← L(q), q′ ← S(c′), u′ ← L(q′), and does the following steps while u′ ≥ u > 0: bump(q′) while u′ > u; then bump(q, q′) if u′ = u. When the loop terminates, c subsumes c′ if and only if u ≤ u′.

(c) Use (b), but set p←F(l=x¯? x:l), and use ((SIG(c) & ∼SIG(x¯)) | SIG(x)) in place of SIG(c). Also modify the detailed test, by inserting ‘if u=x¯ then u ← x’ just after each occurrence of ‘u ← L(q)’.

[The algorithm in (b) was introduced by A. Biere, LNCS 3542 (2005), 59–70, §4.2. “False hits,” in which the detailed test is performed but no actual (self-)subsumption is detected, tend to occur less than 1% of the time in practice.]

375. Let each literal l have another field STAMP(l), initially zero; and let s be a global “time stamp” that is initially zero. To make the test, set s ← s + 1 and σ ← 0; then set STAMP(u) ← s and σ ← σ | SIG(u) for all u such that (l¯u¯) is a clause. If σ ≠ 0, set σ ← σ | SIG(l) and run through all clauses c that contain l, doing the following: If SIG(c) & ∼σ = 0, and if each of c’s literals u ≠ l has STAMP(u) = s, exit with C_1 = c and r = SIZE(c) – 1. If C_1 has thereby been found, set s ← s + 1 and STAMP(ū) ← s for all u ≠ l in c. Then a clause (l¯∨βj) implicitly has j ≤ r in the notation of exercise 369 if and only if βj is a single literal u with STAMP(u) = s.

Given a variable x, test the condition first for l = x; if that fails, try l=x¯.

376. Highest priority is given to the common operations of unit conditioning and pure literal elimination, which are “low-hanging fruit.” Give each variable x two new fields, STATE(x) and LINK(x). A “to-do stack,” containing all such easy pickings, begins at TODO and follows LINKs until reaching Λ. The nonzero states are called FF (forced false), FT (forced true), EQ (eliminated quietly), and ER (eliminated by resolution). Variable x is on the to-do stack only if STATE(x) is FF, FT, or EQ.

Whenever a unit clause (l) is detected, with STATE(|l|) = 0, we set STATE(|l|) ← (l &1? FF: FT), LINK(|l|) ← TODO, and TODO ← |l|. But if STATE(|l|) = (l &1? FT: FF), we terminate, because the clauses are unsatisfiable.

Whenever a literal with TALLY(l¯) = 0 is detected, we do the same thing if STATE(|l|) = 0. But if STATE(|l|) = (l & 1? FT: FF), we simply set STATE(|l|) ← EQ instead of terminating. (In that case TALLY(l) is also 0.)

To clear the to-do stack, we do the following while TODO ≠ Λ: Set x ← TODO and TODO ← LINK(x); if STATE(x) = EQ, do nothing (no erp rule is needed to eliminate x); otherwise set l ← (STATE(x) = FT? x:x¯), output the erp rule l ← 1, and use the doubly linked lists to delete all clauses containing l and to delete l¯ from all clauses. (Those deletions update TALLY and SIZE fields, so they often contribute new entries to the todo stack. Notice that if clause c loses a literal, we must recompute SIG(c). If clause c disappears, we set SIZE(c) ← 0, and never use c again.)

Subsumption and strengthening are next in line. We give each clause c a new field LINK(c), which is nonzero if and only if c appears on the “exploitation stack.” That stack begins at EXP and follows LINKs until reaching the nonzero sentinel value Λ′. All clauses are initially placed on the exploitation stack. Afterwards, whenever a literal l¯ is deleted from a clause c, either during unit conditioning or self-subsumption, we test if LINK(c) = 0; if so, we put c back on the stack by setting LINK(c) ← EXP and EXP ← c.

To clear the exploitation stack, we first clear the to-do stack. Then, while EXP ≠ Λ′, we set c ← EXP, EXP ← LINK(c), and do the following if SIZE(c) ≠ 0: Remove clauses subsumed by c; clear the to-do stack; and if SIZE(c) is still nonzero, strengthen clauses that c can improve, clear the to-do stack, and set TIME(c) ← T (see below).

All of this takes place before we even think about the elimination of variables. But rounds of variable elimination form the “outer level” of computation. Each variable x has yet another field, STABLE(x), which is nonzero if and only if we need not attempt to eliminate x. This field is initially zero, but set nonzero when x is eliminated or its elimination has been abandoned. It is reset to zero whenever a variable is later “touched,” namely when x or x¯ appears in a deleted or self-subsumed clause. (In particular, every variable that appears in a new clause produced by resolution will be touched, because it will appear in at least one of the clauses that were replaced by new ones.)

If a round has failed to eliminate any variables, or if it has eliminated them all, we’re done. But otherwise there’s still work to do, because the new clauses can often be subsumed or strengthened. (Indeed, some of them might actually be duplicates.) Hence two more fields are introduced: TIME(l) for each literal and TIME(c) for each clause, initially zero. Let T be the number of the current elimination round. We set TIME(l) ← T for all literals l in all clauses that are replaced by resolution, and TIME(c) ← T is also set appropriately as mentioned above.

Introduce yet another field, EXTRA(c), initially zero. It is reset to zero whenever TIME(c) ← T , and set to 1 whenever c is replaced by a new clause. For every literal l such that STATE(|l|) = 0 and TIME(l) = T at the end of round T , set EXTRA(c) ← EXTRA(c) + 4 for all clauses c that contain l, and EXTRA(c) ← EXTRA(c) | 2 for all clauses c that contain l¯. Then run through all clauses c for which SIZE(c) > 0 and TIME(c) < T . If SIZE(c) = EXTRA(c) ≫ 2, remove clauses subsumed by c and clear the exploitation stack. Also, if EXTRA(c) &3 ≠ 0, we may be able to use c to strengthen other clauses — unless EXTRA(c) & 1 = 0 and EXTRA(c) ≫ 2 < SIZE(c) – 1. Self-subsumption using l need not be attempted when EXTRA(c) & 1 = 0 unless TIME(l¯) = T and EXTRA(c) ≫ 2 = SIZE(c) – [TIME(l) = T ]. Finally, reset EXTRA(c) to zero (even if TIME(c) = T ). [See Niklas Eén and Armin Biere, LNCS 3569 (2005), 61–75.]

377. Each vertex v of G corresponds to variables v_1, v_2, v_3 in F ; each edge u ––– v corresponds to clauses (ū_1 ∨ v_2), (ū_2 ∨ v_3), (u¯3∨v¯1),(u2∨v¯1),(u3∨v¯2),(u¯1∨v¯3). The longest paths in the dependency digraph for F have the form t1→u2→v3→w¯1 or t1→u¯3→v¯2→w¯1, where t ––– u ––– v ––– w is a walk in G.

[A similar method reduces the question of finding an oriented cycle of length r in a given digraph to the question of finding a failed literal in some dependency digraph. The cycle detection problem has a long history; see N. Alon, R. Yuster, and U. Zwick, Algorithmica 17 (1997), 209–223. So any surprisingly fast algorithm to decide whether or not failed literals exist — that is, faster than n2ω/(ω+1) when m = O(n) and matrix multiplication takes O(nω) — would lead to surprisingly fast algorithms for other problems.]

378. The erp rule l←l∨(l¯1∧…∧l¯q) will change any solution of F \ C into a solution of F . [See M. Järvisalo, A. Biere, and M. Heule, LNCS 6015 (2010), 129–144.]

(In practice it’s sometimes possible to remove tens of thousands of blocked clauses. For example, all of the exclusion clauses (17) in the coloring problem are blocked, as are many of the clauses that arise in fault testing. Yet the author has yet to see a single example where blocked clause elimination is actually helpful in combination with transformations 1–4, which are already quite powerful by themselves.)

379. (Solution by O. Kullmann.) In general, any set F of clauses can be replaced by another set F′, whenever there’s a variable x such that the elimination of x from F yields exactly the same clauses as the elimination of x from F′. In this case the elimination of a has this property. The erp rule a←a∨(b¯∧c¯∧d) is necessary and sufficient.

380. (a) Reverse self-subsumption weakens it to (a ∨ b ∨ c ∨ d), then to (a ∨ b ∨ c ∨ d ∨ e), which is subsumed by (a∨d∨e). [In general one can show that reverse self-subsumption from C leads to a subsumed clause if and only if C is certifiable from the other clauses.]

(b) Again we weaken to (a ∨ b ∨ c ∨ d ∨ e); but now we find this blocked by c

(c) No erp rule is needed in (a), but we need c←c∨(a¯∧b¯) in (b). [Heule, Jäarvisalo, and Biere, LNCS 6397 (2010), 357–371, call this “asymmetric elimination.”]

381. By symmetry, we’ll remove the final clause. (Without it, the given clauses state that x_1 ≤ x_2 ≤ … ≤ xn; with it, they state that all variables are equal.) Assume more generally that, for 1 ≤ j < n, every clause other than (x¯j∨xj+1) that contains x¯j also contains either xn or x¯i for some i < j. For 1 ≤ j < n – 1 we can then weaken (x1∨…∨xj∨x¯n) to (x1∨…∨xj+1∨x¯n). Finally, (x1∨…∨xn−1∨x¯n) can be eliminated because it is blocked by xn–1.

Although we’ve eliminated only one clause, n – 1 erp rules are actually needed to undo the process: x_1 ← x_1 ∨ xn; x2←x2∨(x¯1∧xn); x3←x3∨(x¯1∧x¯2∧xn); ...; xn−1←xn−1∨(x¯1∧…∧x¯n−2∧xn). (Those rules, applied in reverse order, can however be simplified to xj ← xj ∨ xn for 1 ≤ j < n, because x_1 ≤ … ≤ xn in any solution.)

[See Heule, Järvisalo, Biere, EasyChair Proc. in Computing 13 (2013), 41–46.]

382. See M. J. H. Heule, M. Järvisalo, and A. Biere, LNCS 6695 (2011), 201–215.

383. (a) In a learning step, let Φ′ = Φ and Ψ′ = Ψ ∪ C. In a forgetting step, let Φ′ = Φ and Ψ = Ψ′ ∪ C. In a hardening step, let Φ′ = Φ ∪ C and Ψ = Ψ′ ∪ C. In a softening step, let Φ = Φ′ ∪ C and Ψ′ = Ψ ∪ C. In all four cases it is easy to verify that (sat(Φ) ⇔ sat(Φ ∪ Ψ)) implies (sat(Φ) ⇔ sat(Φ′) ⇔ sat(Φ′ ∪ Ψ′)), where sat(G) means “G is satisfiable,” because sat(G ∪ G′) ⇒ sat(G). Thus the assertions are invariant.

(b) Each erp rule allows us to go one step backward, until reaching F.

(c) The first (softening) step is fine, because both Φ = (x) and Φ \ (x) = 1 are satisfiable, and because the erp rule unconditionally makes x true. But the second (learning) step is flawed, because sat(Φ ∪ Ψ) does not imply sat(Φ ∪ Ψ ∪ C) when Φ ∪ Ψ = (x) and C=(x¯). (This example explains why the criterion for learning is not simply ‘sat(Φ) ⇒ sat(Φ ∪ C)’ as it essentially is for softening.)

(d) Yes, because C is also certifiable for Φ ∪ Ψ.

(e) Yes, after softening it. No erp rule is needed, because Φ \ C ⊢ C.

(f) A soft clause can be discarded whether or not it is subsumed. To discard a hard clause that is subsumed by a soft clause, first harden the soft one. To discard a hard C that is subsumed by a hard C′, weaken C and then discard it. (The weakening step is clearly permissible, and no erp rule is needed.)

(g) If C contains x¯ and C′ contains x and C  \  x¯⊆  C′\ x, we can learn the soft clause C ⋄ C′ = C′ \ x, then use it to subsume C′ as in (f).

(h) Forget all soft clauses that contain x or x¯. Then let C_1, ... , Cp be the hard clauses containing x, and C1′,…,Cq′ those containing x¯. Learn all the (soft) clauses Ci⋄Cj′, and harden them, noting that they don’t involve x. Weaken each Ci, with erp rule x←x∨C¯i, and forget it; also weaken and forget each Cj′, with erp rule x←x∧Cj′. (One can show that either of the erp rules in (161) would also suffice.)

(i) Whenever Φ∪Ψ is satisfiable, so is Φ∪Ψ∪{(x∨z), (y ∨z), (x¯∨y¯∨z¯)}, because we can always set z←x¯∨y¯.

[Reference: M. Järvisalo, M. Heule, and A. Biere, LNCS 7364 (2012), 355–370. Notice that, by exercise 368, parts (f) and (h) justify the use of unit conditioning.]

384. Whenever we have a solution to Φ \ C that falsifies C, we will show that Φ is satisfied by making l true; hence softening C is permissible, with erp rule l←l∨C¯.

To prove that claim, notice that a problem could arise only in a hard clause C′that contains l¯. But if all other literals of C′ are false in the given solution, then all literals of C ⋄ C′ are false, contradicting the assumption that (Φ \  C)∧C⋄C′¯  ⊢1 ∊.

(Such clauses C are “resolution certifiable” with respect to Φ\C. Blocked clauses are a very special case. Similarly, we can safely learn any clause that is resolution certifiable with respect to Φ ∪ Ψ.)

385. (a) True, because C¯ ∧l ⊢1 ∊ when l ∈ C.

(b) 1¯ is implied, not certifiable; 1¯2 is certifiable, not absorbed; 1¯23 is absorbed.

(c, d) If C is any clause and l is any literal, then F ∧C¯ ⊢1 l implies F′ ∧C¯ ⊢1 l, because unit propagation in F carries over to unit propagation in F′.

386. (a) The trail contained exactly score(F, C, l) literals when decision l¯ was made at level d during the helpful round. The clause learned from the ensuing conflict causes at least one new literal to be implied at level d′ < d.

(b) The score can’t decrease when F grows.

(c) Each l ∈ C needs at most n helpful rounds to make score(F, C, l) = ∞.

(d) Suppose, for example, F=(a∨d¯)∧(a∨b∨e∨l)∧(a¯∨c)∧(b¯)∧(c∨d∨e¯∨l) and C = (a ∨ b ∨ c ∨ d ∨ l). The helpful sequences of decisions are (a¯,c¯,l¯),(c¯,l¯),(d¯,a¯,c¯,l¯), (d¯,c¯,l¯), and they occur with probabilities 110 16 14, 110 14, 110 18 16 14, 110 18 14.

In general if a decision is to be made and j elements of C¯ are not yet in the trail, the probability that suitable decisions will be made at random is at least
f(n,j)=min(j−12nf(n−1,j−1),j−22(n−1)f(n−2,j−2),…,                                                                                                          12(n−j+2)f(n−j+1,1),12(n−j+1))=(j−1)!2jnj¯.

(e) The waiting time to absorb each clause Ci is upper-bounded by a geometric distribution whose mean is ≤ 4n|Ci|, repeated at most |Ci|n times.

References: K. Pipatsrisawat and A. Darwiche, Artif. Intell. 175 (2011), 512–525; A. Atserias, J. K. Fichte, and M. Thurley, J. Artif. Intell. Research 40 (2011), 353–373.

387. We may assume that G and G′ have no isolated vertices. Letting variable vv′mean that v corresponds to v′, we need the clauses (uv′¯∨vv′¯) for u < v and (vu′¯∨vv′¯) for u′ < v′. Also, for each u < v with u ––– v in G, we introduce auxiliary variables uu′vv′ for each edge u′ ––– v′ in G′, with clauses (uu′vv′¯∨uu′)∧(uu′vv′¯∨vv′) ∧ (∨ {uu′vv′ | u′ ––– v′ in G′}). The variables vv′ and uu′vv′ can be restricted to cases where degree(u) ≤ degree(u′) and degree(v) ≤ degree(v′).

388. (a) Can the complete graph Kk be embedded in G? (b) Can G be embedded in the complete k-partite graph Kn,...,n, where G has n vertices? (c) Can the cycle Cn be embedded in G?

389. This is similar to a graph embedding problem, with G′ the 4 × 4 (king ∪ knight) graph and with G defined by edges T ––– H, H ––– E, E ––– ⊔, ... , N ––– G; however, we allow v′ = w′ when v ≠ w, and labels must match. The adjacent Ms can be avoided by changing ‘PROGRAMMING’ to either ‘PROGRAMXING’ or ‘PROGRAXMING’.

Algorithm C needs fewer than 10 megamems to find the first solution below. Furthermore, if the blank space can also be moved, the algorithm will rather quickly also find solutions with just five knight moves (the minimum), or 17 of them (the max):

390. Let d(u, v) be the distance between vertices u and v. Then d(v, v) = 0 and
d(u,v)≤j+1 ⇔d(u,w)≤j  for  some w∈N(v)={w|w—v}

if u ≠ v. In parts (a), (d), we introduce variables vj for each vertex v and 0 ≤ j ≤ k. In part (c) we do this for 0 ≤ j < n. But parts (b), (e), (f) use just n variables, {v | v ∈ V }.

(a) Clauses (s0)∧∧v∈V\s  ((v¯0) ∧∧j=0k−1(v¯j+1 ∨∨w∈N(v) wj)) are satisfied only if vj ≤ [d(s, v) ≤ j]; hence the additional clause (tk) is also satisfied only if d(s, t) ≤ k. Conversely, if d(s, t) ≤ k, all clauses are satisfied by setting vj ← [d(s, v) ≤ j ].

(b) There’s a path from s to t if and only if there’s a subset H ⊆ V such that s ∈ H, t ∈ H, and every vertex v of the induced graph G | H has degree 2 – [v = s] – [v = t]. [The vertices on a shortest path from s to t yield one such H. Conversely, given H, we can find vertices vj ∈ H such that s = v_0 ––– v_1 –––…––– vk = t.]

We can represent that criterion via clauses on the binary variables v = [v ∈ H ] by asserting (s) ∧ (t), together with clauses to ensure that Σ(v) = 2 – [v = s] – [v = t] for all v ∈ H, where Σ(v) = Σw∈N(v) w is the degree of v in G | H. The number of such clauses for each v is at most 6|N(v)|, because we can append v¯ to each clause of (18) and (19) with r = 2, and |N(v)| additional clauses will rule out Σ(v) < 2. Altogether there are O(m) clauses, because Σv∈V |N(v)| = 2m.

[Similar but simpler alternatives, such as (i) to require Σ(v) ∈ {0, 2} for all v ∈ V \{s, t}, or (ii) to require Σ(v) ≥ 2 for all v ∈ H \{s, t},do not work: Counterexamples are (i) s t and (ii) s t. Another solution, more cumbersome, associates a Boolean variable with each edge of G.]

(c) Let s be any vertex; use (a) with k = n – 1, plus (vn–1) for all v ∈ V \ s.

(d) Clauses (s0)∧∧j=0k−1 ∧v∈V  ∧w∈N(v) (v¯j∨wj+1) are satisfied only if we have vj ≥ [d(s, v) ≤ j ]; hence the additional unit clause (t¯k) cannot also be satisfied when d(s, t) ≤ k. Conversely, if d(s, t) > k we can set vj ← [d(s, v) ≤ j].

(e) (s)∧(∧v∈V  ∧w∈N(v) (v¯∨w))∧(t¯).

(f) Letting s be any vertex, use (s)∧(∧v∈V  ∧w∈N(v) (v¯∨w))∧(∨v∈V\s v¯).

[Similar constructions work with digraphs and strong connectivity. Parts (d)–(f) of this exercise were suggested by Marijn Heule. Notice that parts (a) and (c)–(f) construct renamed Horn clauses, which work very efficiently (see exercise 444).]

391. (a) Let d – 1 = (ql–1 ...q_0)2. To ensure that (xl–1 ...x_0)2 < d we need the clauses (x¯i∨∨{x¯j | j>i, qj=1})52 whenever qi = 0. The same holds for y.

To enforce x ≠ y, introduce the clause (al–1 ∨ … ∨ a_0) in auxiliary variables al–1 ...a_0, together with (a¯j∨xj∨yi)∧(a¯j∨x¯j∨y¯i) for 0 ≤ j < l (see (172)).

(b) Now x ≠ y is enforced via clauses of length 2l, which state that we don’t have x = y = k for 0 ≤ k < d. For example, the appropriate clause when l = 3 and k = 5 is (x¯2∨y¯2∨x1∨y1∨x¯0∨y¯0).

(c) Use the clauses of (b) for 0 ≤ k < 2d – 2l, plus clauses of length 2l – 2 for d ≤ k < 2l stating that we don’t have (xl–1 ...x_1)2 = (yl–1 ...y_1)2 = k – 2l–1. (The encodings in (b) and (c) are identical when d = 2l.)

[See A. Van Gelder, Discrete Applied Mathematics 156 (2008), 230–243.]

392. (a) [Puzzle (ii) was introduced by Sam Loyd in the Boston Herald, 13 November 1904; page 27 of his Cyclopedia (1914) states that he’d created a puzzle like (i) at age 9! Puzzle (iv) is by H. E. Dudeney, Strand 42 (1911), 108, slightly modified. Puzzle (iii) is from the Grabarchuks’ Big, Big, Big Book of Brainteasers (2011), #196; puzzle (v) was designed by Serhiy A. Grabarchuk in 2015.]

(b) [Puzzle (vi) is an instance of the odd-even transposition sort, exercise 5.3.4– 37. Eight order-reversing connections would be impossible with only eight columns, instead of the nine in (vii), because the permutation has too many inversions.]

(c) Let dj=Σi=1j (|Ti|−1) and d = dt. We introduce variables vi for 1 ≤ i ≤ d, and the following clauses for 1 ≤ j ≤ t and dj–1 < i ≤ dj: (v¯i,∨v¯i) for 1 ≤ i′ ≤ dj–1; the clauses of answer 390(b) on variables vi, where s is the (i – dj–1)th element of Tj and t is the last element. These clauses ensure that the sets Vj = {v | vdj–1+1∨...∨vdj} are disjoint, and that Vj contains a connected component Sj ⊇ Tj.

We also assert (v¯i) for 1 ≤ i ≤ d, whenever Tj is a singleton set {v}.

[For the more general “Steiner tree packing” problem, see M. Grötschel, A. Martin, and R. Weismantel, Math. Programming 78 (1997), 265–281.]

393. A construction somewhat like that of answer 392(c) can be used with five different 8 × 8 graphs, one for the moves of each white-black pair Sj. But we need to keep track of the edges used, not vertices, in order to prohibit edges that cross each other. Additional clauses will rule that out.

394. Call these clauses langford′"(n). [Steven Prestwich described a similar method in Trends in Constraint Programming (Wiley, 2007), 269–274.] Typical results are:

variables clauses Algorithm D Algorithm L Algorithm C

langford′"(9)

206

1157

131 Mμ

18 Mμ

22 Mμ

(UNSAT)

langford′"(13)

403

2935

1425 Gμ

44 Gμ

483 Gμ

(UNSAT)

langford′"(16)

584

4859

713 Kμ

42 Mμ

343 Kμ

(SAT)

langford′"(64)

7352

120035

(huge)

(big)

71 Mμ

(SAT)

395. The color of each vertex v gets binary axiom clauses (v¯j+1∨vj) for 1 ≤ j < d–1, as in (164). And for each edge u ––– v in the graph, we want d clauses (u¯j−1∨uj∨v¯j−1∨vj) for 1 ≤ j ≤ d, omitting ū^0 and v¯0 when j = 1, ud and vd when j = d.

[The surprising usefulness of order encoding in graph coloring was first noticed by N. Tamura, A. Taga, S. Kitagawa, and M. Banbara in Constraints 14 (2009), 254–272.]

396. First we have (x¯j+1∨xj) and x^j+1¯∨x^j for 1 ≤ j < d. Then we have “channeling” clauses to ensure that j ≤ x < j + 1 ⇔ jπ ≤ xπ < jπ + 1 for 0 ≤ j < d:

(x¯j∨xj+1∨x^jπ)∧(x¯j∨xj+1∨x^jπ+1¯)∧(x^jπ¯∨x^jπ+1∨xj)∧(x^jπ¯∨x^jπ+1∨x¯j+1).

(These clauses should be either shortened or omitted in boundary cases, because x^0 and x^0 are always true, while xd and x^d are always false. We obtain 6d–8 clauses for each x.)

With such clauses for every vertex of a graph, together with clauses based on adjacent vertices and cliques, we obtain encodings for n-coloring the n × n queen graph that involve 2(n^3 – n^2) variables and 53n4+4n3+O(n2) clauses, compared to n^3 – n^2 variables and 53n4−n3+O(n2) clauses with single cliques and (162) alone. Typical running times with Algorithm C and single cliques are 323 Kμ, 13.1Mμ, 706 Gμ for n = 7, 8, 9; with double clique-ing they become 252 Kμ, 1.97 Mμ, 39.8Gμ, respectively.

The double clique hints turn out to be mysteriously ineffective when π is the standard organ-pipe permutation (0π, 1π,..., (d–1)π) = (0, 2, 4,..., 5, 3, 1) instead of its inverse. Random choices of π when n = 8 yielded significant improvement almost half the time, in the author’s experiments; but they had negligible effect in 1/3 of the cases.

Notice that the example π for d = 4 yields x1=x¯0, x3=x3, x^1=x¯2,x^3=x1. Hence the direct encoding is essentially present as part of this redundant representation, and the hints (u¯3∨v¯3)∧(u1∨v1)∧(u^3¯∨v^3¯)∧(u^1∨v^1) for 2-cliques {u, v} are equivalent to (16). But the hints (u2∨v2∨w2)∧(u¯2∨v¯2∨w¯2)∧(u^2∨v^2∨w^2)∧(u^2¯∨v^2¯∨w^2¯) that apply when {u, v, w} is a triangle give additional logical power.

397. There are (p – 2)d binary clauses (y¯ji+1∨yji) for 1 ≤ i < p – 1, together with the (2p – 2)d clauses (x¯ij∨xij+1∨yji)∧(x¯i−1j∨xi−1j+1∨y¯ji) for 1 ≤ i < p, all for 0 ≤ j < d. The hint clauses (x0p−1∨…∨xp−1p−1)∧(x¯0d−p+1∨…∨x¯p−1d−p+1) are also valid.

(This setup corresponds to putting p pigeons into d holes, so we can usually assume that p ≤ d. If p ≤ 4 it is better to use (p2)d clauses as in exercise 395. Notice that we obtain an interesting representation of permutations when p = d. In that case y is the inverse permutation; hence (2d – 2)p additional clauses corresponding to yj = i ⇒ xi = j are also valid, as well as two hint clauses for y.)

A related idea, but combined with direct encoding of the x’s, was presented by I. Gent and P. Nightingale in Proceedings of the International Workshop on Modelling and Reformulating Constraint Satisfaction Problems 3 (2004), 95–110.

398. We could construct (3p – 4)d binary clauses that involve yji, as in exercise 397. But it’s better just to have (3p–6)d clauses for the at-most-one constraints x_0k +x_1k + … + x(p–1)k ≤ 1, 0 ≤ k < d.

399. (a) d^2 –t preclusion clauses (binary); or 2d support clauses (total length 2(d+t)).

(b) If unit propagation derives v¯j from (u¯i∨v¯j), we knew ui; hence (17) gives ūi for all i′ ≠ i, and v¯j follows from the support clause that contains it.

(c) If unit propagation derives v¯j from its support clause, we knew ūi for all i ≠ j; hence (15) gives uj, and v¯j follows from (16). Or if unit propagation derives ui from that support clause, we knew vj and ūi for all i′ ∉ {i, j}; hence ūj from (16), ui from (15).

(d) A trivial example has no legal pairs; then unit propagation never gets started from binary preclusions, but the (unit) support clauses deduce all. A more realistic example has d = 3 and all pairs legal except (1, 1) and (1, 2), say; then we have (15)∧(17)∧(u¯1∨v¯1)∧(u¯1∨v¯2)∧(v¯3)⊬1 u¯1 but (15) ∧ (17) ∧ (ū_1 ∨ v_3) ∧ (v¯3) ⊢1 ū_1.

[Preclusion was introduced by S. W. Golomb and L. D. Baumert, JACM 12 (1965), 521–522. The support encoding was introduced by I. P. Gent, European Conf. on Artificial Intelligence 15 (2002), 121–125, based on work of S. Kasif, Artificial Intelligence 45 (1990), 275–286.]

400. This problem has n variables q_1, ... , qn with n values each; thus there are n^2 Boolean values, with qij = [qi = j] = [there’s a queen in row i and column j]. The constraint between qi and qj is that qi ∉ {qj,qj + i – j, qj – i + j}; so it turns out that there are n at-least-one clauses, plus (n^3–n^2)/2 at-most-one clauses, plus either n^3–n^2 support clauses or n3−n2+(n3) preclusion clauses. In this problem each support clause has at least n – 2 literals, so the support encoding is much larger.

Since the problem is easily satisfiable, it makes sense to try WalkSAT. When n = 20, Algorithm W typically finds a solution from the preclusion clauses after making fewer than 500 flips; its running time is about 500 Kμ, including about 200 Kμ just to read the input. With the support clauses, however, it needs about 10 times as many flips and consumes about 20 times as many mems, before succeeding.

Algorithm L is significantly worse: It consumes 50 Mμ with preclusion clauses, 11 Gμ with support clauses. Algorithm C is the winner, with about 400 Kμ (preclusion) versus 600 Kμ (support).

Of course n = 20 is pretty tame; let’s consider n = 100 queens, when there are 10,000 variables and more than a million clauses. Algorithm L is out of the picture; in the author’s experiments, it showed no indication of being even close to a solution after 20 Tμ! But Algorithm W solves that problem in 50 Mμ, via preclusion, after making only about 5000 flips. Algorithm C wins again, polishing it off in 29 Mμ. With the support clauses, nearly 100 million literals need to be input, and Algorithm W is hopelessly inefficient; but Algorithm C is able to finish after about 200 Mμ.

The preclusion clauses actually allow us to omit the at-most-one clauses in this problem, because two queens in the same row will be ruled out anyway. This trick improves the run time when n = 100 to 35 Mμ for Algorithm W.

We can also append support clauses for the columns as well as the rows. This idea roughly halves the search space, but it gives no improvement because twice as many clauses must be handled. Bottom line: Support clauses don’t support n queens well.

(However, if we seek all solutions to the n queens problem instead of stopping with the first one, using a straightforward extension of Algorithm D (see exercise 122), the support clauses proved to be definitely better in the author’s experiments.)

401. (a) yj = x2j–1. (b) zj = x3j–1. In general w = ⌊(x + a)/b⌋ ⇔ wj = xbj–a.

402. (a) ∧j=1⌊d/2⌋ (x¯2j−1∨x2j); (b) ∧j=1⌊d/2⌋ (x¯2j−1∨x2j−1); omit x¯0 and xd.

403. (a) ∧j=1d−1 (x¯j∨y¯j∨zj); (b) ∧j=1d−1 ((x¯j∨zj)∧(y¯j∨zj)); (c) ∧j=1d−1 ((x¯j∨z¯j)∧(y¯j∨z¯j)); (d) ∧j=1d−1 (xj∨yj∨z¯j).

404. ∧j=0d−a (x¯j∨xj+a∨y¯j∨yj+a). (As usual, omit literals with superscript 0 or d.)

405. (a) If a < 0 we can replace ax by (−a)x¯ and c by c + a – ad, where x¯ is given by (165). A similar reduction applies if b < 0. Cases with a, b, or c = 0 are trivial.

(b) We have 13x+8y¯≤63 ⇔ not 13x+8y¯≥64 ⇔ not (P_0 or ... or Pd–1) ⇔ not P_0 and ... and not Pd–1, where Pj = ‘x ≥ j and y¯≥⌈(64−13j)/8⌉’. This approach yields ∧j=07(x¯j∨y8−⌈(64−13j)/8⌉), which simplifies to (x¯1∨y1)∧(x¯2∨y3)∧(x¯3∨y4)∧(x¯4∨y6)∧(x¯5). (Notice that we could have defined Pj=‘y¯≥j and x ≥ ⌈(64 – 8j)/13⌉’ instead, thereby obtaining the less efficient encoding (x¯5)∧(y7∨x¯5)∧(y6∨x¯4)∧(y5∨x¯4)∧(y4∨x¯3)∧(y3∨x¯2)∧(y2∨x¯2)∧(y1∨x¯1); it’s better to discriminate on the variable with the larger coefficient.)

(c) Similarly, 13x¯+8y≤90 gives (x5∨y¯7)∧(x4∨y¯5)∧(x3∨y¯4)∧(x2∨y¯2)∧(x1). (The (x, y) pairs legal for both (b) and (c) are (1, 1), (2, 3), (3, 4), (4, 6).)

(d) ∧j=max(0,⌈c+1−b(d−1))/a⌉)min(d−1,⌈(c+1)/a⌉)(x¯j∨y¯⌈(c+1−aj)/b⌉), when a ≥ b > 0 and c ≥ 0.

406. (a) (∧j=⌈(a+1)/(d−1)⌉⌊a+1⌋ (x¯j∨y¯⌈(a+1)/j⌉))∧(∧j=⌈(a+1)/(d−1)⌉⌈a+1⌉−1  (x¯⌈(a+1)/j⌉∨y¯j)).

(b) (∧j=l+1⌊a−1⌋+1 (x¯j∨y⌊(a−1)/(j−1)⌋+1))∧(∧j=l+1⌈a−1⌉  (x⌊(a−1)/j−1⌋+1∨yj)) ∧ (xl) ∧ (yl), where l = ⌊ (a – 1)/(d – 1)⌋ + 1. [Both formulas belong to 2SAT.]

407. (a) We always have ⌊x/2⌋+⌈x/2⌉=x,⌊x/2⌋+⌊y/2⌋≤x+y2≤⌊x/2⌋+⌊y/2⌋+1, and ⌈x/2⌉+⌈y/2⌉−1≤x+y2≤⌈x/2⌉+⌈y/2⌉. (Similar reasoning proves the correctness of Batcher’s odd-even merge network; see Eq. 5.3.4–(3).)

(b) Axiom clauses like (164) needn’t be introduced for u and v, or even for z; so they aren’t counted here, although they could be added if desired. Let ad = d^2 – 1 be the number of clauses in the original method; then the new method has fewer clauses when a⌈d/2⌉ + a⌊d/2⌋+1 + 3(d – 2) < ad, namely when d ≥ 7. (The new method for d = 7 involves 45 clauses, not 48; but it introduces 10 new auxiliary variables.) Asymptotically, we can handle d = 2t + 1 with 3t2t + O(2t) = 3d lg d + O(d) clauses and d lg d + O(d) auxiliary variables.

(c) x + y ≥ z ⇔ (d – 1 – x)+(d – 1 – y) ≤ (2d – 2 – z); so we can use the same method, but complemented (namely with xj↦x¯d−j,yj↦y¯d−j,zj↦z¯2d−1−j).

[See N. Tamura, A. Taga, S. Kitagawa, and M. Banbara, Constraints 14 (2009), 254–272; R. Asín, R. Nieuwenhuis, A. Oliveras, and E. Rodríguez-Carbonell, Constraints 16 (2011), 195–221.]

408. (a) No; makespan 11 is best, achievable as follows (or via left-right reflection):

(b) If j is the last job processed by machine i, that machine must finish at time ≤ Σk=1n wik+Σk=1mwkj−wij, because j uses some other machine whenever i is idle. [See D. B. Shmoys, C. Stein, and J. Wein, SICOMP 23 (1994), 631.]

(c) Clearly 0 ≤ sij ≤ t – wij. And if ij ≠ i′j′ but i = i′ or j = j′, we must have either sij + wij ≤ si′j′ or si′j′ + wi′j′ ≤ sij whenever wijwi′j′ ≠ 0.

(d) When wij > 0, introduce Boolean variables sijk for 1 ≤ k ≤ t – wij, with the axiom clauses (s¯ijk+1∨sijk) for 1 ≤ k < t – wij. Then include the following clauses for all relevant i, j, i′, and j′ as in (c): For 0 ≤ k ≤ t + 1 – wij – wi′j′, assert (p¯iji′j′∨sij−k∨si′j′k+wij) if ij < i′j′ or (p¯i′j′ij∨sij−k∨si′j′k+wij) if ij > i′j′, omitting s¯ij0 in the first of these ternary clauses and omitting si′j′t+1−wi′j′ in the last.

[This method, introduced by N. Tamura, A. Taga, S. Kitagawa, and M. Banbara in Constraints 14 (2009), 254–272, was able to solve several open shop scheduling problems in 2008 that had resisted attacks by all other approaches.]

Since the left-right reflection of any valid schedule is also valid, we can also save a factor of two by arbitrarily choosing one of the p variables and asserting (piji′j′).

(e) Any schedule for W and T yields a schedule for ⌊W/k⌋ and ⌈T/k⌉, if we examine time slots 0, k, 2k, ... . [With this observation we can narrow down the search for an optimum makespan by first working with simpler problems; the number of variables and clauses for ⌊W/k⌋ and T/k is about 1/k times the number for W and T , and the running time also tends to obey this ratio. For example, the author solved a nontrivial 8×8 problem by first working with ⌊W/8⌋ and getting the respective results (U, S, U) for t = (128, 130, 129), where ‘U’ means “unsatisfiable” and ‘S’ means “satisfiable”; running times were about (75, 10, 1250) megamems. Then with ⌊W/4⌋ it was (S, U, U) with t = (262, 260, 261) and runtimes (425, 275, 325); with ⌊W/2⌋ it was (U, S, U) with t = (526, 528, 527) and runtimes (975, 200, 900). Finally with the full W it was (U, S, S) with t = (1058, 1060, 1059) and runtimes (2050, 775, 300), establishing 1059 as the optimum makespan while doing most of the work on small subproblems.]

Notes: Further savings are possible by noting that any clauses learned while proving that t is satisfiable are valid also when t is decreased. Difficult random problems can be generated by using the following method suggested by C. Guéret and C. Prins in Annals of Operations Research 92 (1999), 165–183: Start with work times wij that are as near equal as possible, having constant row and column sums s. Then choose random rows i ≠ i′ and random columns j ≠ j′, and transfer δ units of weight by setting wij ← wij – δ, wi′j ← wij′ + δ, wij′ ← wij′ + δ, wi′j′ ← wi′j′ – δ, where δ ≤ wij and δ ≤ wi′j′; this operation clearly preserves the row and column sums. Choose δ at random between p · min{wij,wi′j′} and min {wij,wi′j′}, where p is a parameter. The final weights are obtained after making r such transfers. Guéret and Prins suggested choosing r = n^3, and p = .95 for n ≥ 6; but other choices give useful benchmarks too.

409. (a) If S ⊆ {1,...,r}, let ΣS = Σj∈S aj. We can assume that job n runs on machines 1, 2, 3 in that order. So the minimum makespan is 2w_2n + x, where x is the smallest ΣS that is ≥ ⌈(a_1 + … + ar)/2⌉. (The problem of finding such an S is well known to be NP-hard [R. M. Karp, Complexity of Computer Computations (New York: Plenum, 1972), 97–100]; hence the open shop scheduling problem is NP-complete.)

(b) Makespan w_2n + w_4n is achievable if and only if ΣS = (a_1 + … + ar)/2 for some S. Otherwise we can achieve makespan w_2n + w_4n + 1 by running jobs 1, ... , n in order on machine 1 and letting s3(n–1) = 0, s_4n = w_2n; also s_2j = w_2n + w_4n, if machine 1 is running job j at time w_2n. The other jobs are easily scheduled.

(c) ⌊3n/2⌋ – 2 time slots are clearly necessary and sufficient. (If all row and column sums of W are equal to s, can the minimum makespan be ≥32s?)

(d) The “tight” makespan s is always achievable: By renumbering the jobs we can assume that aj ≤ bj for 1 ≤ j ≤ k, aj ≥ bj for k < j ≤ n, b_1 = max{b_1,...,bk}, an = max{ak+1,...,an}. Then if bn ≥ a_1, machine 1 can run jobs (1,...,n) in order while machine 2 runs (n, 1,...,n – 1); otherwise (2,...,n, 1) and (1,...n) suffice.

If a_1 + … + an ≠ b_1 + … + bn, we can increase an or bn to make them equal. Then we can add a “dummy” job with an+1 = bn+1 = max{a_1 + b_1,...,an + bn}. – s, and obtain an optimum schedule in O(n) steps as explained above.

Results (a), (b), (d) are due to T. Gonzalez and S. Sahni, who introduced and named the open shop scheduling problem in JACM 23 (1976), 665–679. Part (c) is a subsequent observation and open problem due to Gonzalez (unpublished).

410. Using half adders and full adders as we did in (23) allows us to introduce intermediate variables wj such that (x_2x_1x_0)2 +(x_2x_1x_000)2 +(x_2x_1x_0000)2 +(y¯2y¯1y¯0000)2 ≤ (w_7w_6 ...w_0)2, and then to require (w¯7)∧(w¯6). In slow motion, we successively compute (c_0z_0)2 ≥ x_0 + x_1, (c1z1)2≥x0+x1+y¯0, (c_2z_2)2 ≥ c_0 + z_1, (c3z3)2≥x1+x2+y¯1, (c_4z_4)2 ≥ c_1 + c_2 + z_3, (c5z5)2≥x1+y¯2+c3, (c_6z_6)2 ≥ c_4 + z_5, (c_7z_7)2 ≥ c_5 + c_6; then w_7w_6 ...w_0 = c_7z_7z_6z_4z_2z_0x_1x_0. In slower motion, each step (cizi)2 ≥ u + v expands to zi ≥ u ⊕ v, ci ≥ u ∧ v; each step (cizi)2 ≥ t + u + v expands to si ≥ t ⊕ u, pi ≥ t ∧ u, zi ≥ v ⊕ s, qi ≥ v ∧ s, ci ≥ pi ∨ qi. And at the clause level, t≥u∧v⇔(t∨u¯∨v¯); t≥u∨v⇔(t∨u¯)∧(t∨v¯); t≥u⊕v⇔(t∨u¯∨v)∧(t∨u∨v¯). [Only about half of (24) is needed when inequalities replace equalities. Exercise 42 offers improvements.]

We end up with 44 binary and ternary clauses; 10 of them can be omitted, because z_0, z_2, z_4, z_6, and z_7 are pure literals, and the clause for c_7 can be omitted if we simply require c_5 = c_6 = 0. But the order encoding of exercise 405 is clearly much better. The log encoding becomes attractive only with larger integers, as in the following exercise. [See J. P. Warners, Information Processing Letters 68 (1998), 63–69.]

411. Use m + n new variables to represent an auxiliary number w = (wm+n ...w_1)2. Form clauses as in exercise 41 for the product xy = w; but retain only about half of the clauses, as in answer 410. The resulting 9mn – 5m – 10n clauses are satisfiable if w = xy; and we have w ≥ xy whenever they are satisfiable. Now add 3m +3n – 2 further clauses as in (169) to ensure that z ≥ w. The case z ≤ xy is similar.

412. Mixed-radix representations are also of interest in this connection. See, for example, N. Eén and N. Sörensson, J. Satisfiability, Bool. Modeling and Comp. 2 (2006), 1–26; T. Tanjo, N. Tamura, and M. Banbara, LNCS 7317 (2012), 456–462.

413. There’s only one, namely ∧σ_1,...,σn∈{–1,1}(σ_1x_1 ∨σ_1y_1 ∨...∨σnxn ∨σnyn). Proof: Some clause must contain only positive literals, because f(0,..., 0) = 0. This clause must be (x_1 ∨ y_1 ∨…∨ xn ∨ yn); otherwise it would be false in cases where f is true. A similar argument shows that every clause (σ_1x_1 ∨ σ_1y_1 ∨…∨ σnxn ∨ σnyn) must be present. And no clause for f can contain both xj and y¯j, or both x¯j and yj.

414. Eliminating first an–1, then an–2, etc., yields 2n–1 clauses. (The analogous result for x_1 . . . xn < y_1 . . . yn is 2n + 2n–1 + 1. A preprocessor will probably eliminate an–1.)

415. Construct clauses for 1 ≤ k ≤ n that represent ‘ak–1 implies xk < yk + ak’:

(a¯k−1∨∨j=1d−1(x¯kj∨ykj))∧(a¯k−1∨ak∨∨j=0d−1(x¯kj∨ykj+1)),omitting⁢   x¯k0⁢    and⁢   ykd;
also omit ā_0. For the relation x_1 . . . xn ≤ y_1 . . . yn we can omit the d clauses that contain the (pure) literal an. But for x_1 . . . xn < y_1 . . . yn, we want an = 0; so we omit an and the d – 1 clauses (a¯n−1∨x¯nj∨ynj). [The clauses (169) are due to K. Sakallah, Handbook of Satisfiability (2009), Chapter 10, (10.32).]

416. The other clauses are ∧i=1m((ui∨v¯i∨a¯0)∧(u¯i∨vi∨a¯0)) and (a_0 ∨ a_1 ∨ ... ∨ an). [See A. Biere and R. Brummayer, Proceedings, International Conference on Formal Methods in Computer Aided Design 8 (IEEE, 2008), 4 pages [FMCAD08].]

417. The four clauses (s¯∨t¯∨u)∧(s¯∨t∨v)∧(s∨t¯∨u¯)∨(s∨t∨v¯) ensure that s is true if and only if t? u: v is true. But we need only the first two of these, as in (173), when translating a branching program, because the other two are blocked in the initial step. Removing them makes the other two blocked on the second step, etc.

418. A suitable branching program for hn when n = 3, beginning at I_11, is I11=(1¯?21:22), I21=(2¯?31:32), I22=(2¯?32:33), I31=(3¯?0:42), I32=(3¯?42:43), I33=(3¯?43:1), I42=(1¯?0:1), I43=(2?0:1). It leads via (173) to the following clauses for row i, 1 ≤ i ≤ m: (ri,1,1); (r¯i,k,j∨xik∨ri,k+1,j)∧(r¯i,k,j∨x¯ik∨ri,k+1,j+1), for 1 ≤ j ≤ k ≤ n; (r¯i,n+1,1)∧(ri,n+1,n+1) and (r¯i,n+1,j+1∨xij) for 1 ≤ j < n. Also the following clauses for column j, 1 ≤ j ≤ n: (ci,1,1); (c¯j,k,i∨xkj∨cj,k+1,i)∧(c¯j,k,i∨x¯kj∨cj,k+1,i+1), for 1 ≤ i ≤ k ≤ m; (c¯j,m+1,1)∧(cj,m+1,m+1) and (c¯j,m+1,i+1∨xij) for 1 ≤ i < m.

419. (a) There are exactly n–2 solutions: xij = [j = 1][i ≠ m– 1] + [j = 2][i = m– 1] + [j = k][i = m–1], for 2 < k = n.

(b) There are exactly m–2 solutions: x¯ij=[j>1][i=m−1]+[j=1][i=m−2]+[j=1][i=k], for 1 = k < m–2 or k = m.

420. Start via (24) with (x¯1∨x2∨s)∧(x1∨x¯2∨s) ∧ (x1∨x2∨s¯)∧(x¯1∨x¯2∨s¯); (x1∨c¯)∧(x2∨c¯)∧(x¯1∨x¯2∨c); (s¯∨x3∨t)∧(s∨x¯3∨t)∧(s∨x3∨t¯)∧(s¯∨x¯3∨t¯); (s∨c¯′)∧(x3∨c¯′)∧(s¯∨x¯3∨c′); (c¯)∧(c¯′). Propagate (c¯) and (c¯′), obtaining (x¯1∨x¯2)∧(s¯∨x¯3); remove subsumed clauses (x¯1∨x¯2∨s¯), (s¯∨x¯3∨t¯); remove blocked clause (s∨x3∨t¯); remove clauses containing the pure literal t; rename s to a_1.

421. Start via (173) with (a¯5∨x1∨a4)∧(a¯5∨x¯1∨a4)∧(a¯5∨x¯1∨a3)∧(a¯4∨x¯2∨a2)∧(a¯3∨x2∨a2) ∧ (a¯3∨x¯2)∧(a¯2∨x¯3)∧(a5). Propagate (a_5).

422. (a) x_1 implies x¯2, then a_1, then x¯3; x_2 implies x¯1, then a_1, then x¯3.

(b) x_1 implies a_3, then x¯2, then a_2, then x¯3; x_2 implies ā_3, then x¯1, a_4, a_2, x¯3.

423. No; consider x_1? (x_2? x_3: x_4): (x_2? x_4: x_3) with L=(x¯3)∧(x¯4). (But Abío, Gange, Mayer-Eichberger, and Stuckey have shown [LNCS 9676 (2016), 1–17] that weak forcing is always achieved if (āj∨al ∨ ah) is added to (173). Furthermore, a forcing encoding can always be constructed, via the extra clauses defined in exercise 436. Notice that, in the presence of failed literal tests, weak forcing corresponds to forcing.)

424. The clause 1¯3¯4¯ is redundant (in the presence of 1¯2¯3¯ and 23¯4¯); it cannot be omitted, because {2¯3¯,23¯,12}⊬1 3¯. The clause 23¯4¯ is also redundant (in the presence of 1¯3¯4¯ and 12); it can be omitted, because {1¯4¯,34,1}⊢1 4¯, {1¯3¯,34,1}⊢1 3¯, and {1¯2¯,1¯,12}⊢12.

425. If x is in the core, F ⊢1 x, because Algorithm 7.1.1C does unit propagation. Otherwise F is satisfied when all core variables are true and all noncore variables are false.

426. (a) True. Suppose the clauses involving am are (am ∨ αi) for 1 ≤ i ≤ p and (ām ∨ βj) for 1 ≤ j ≤ q; then G contains the pq clauses (αi ∨ βj) instead. If F | L ⊢1 l we want to prove that G | L ⊢1 l. This is clear if unit propagation from F | L doesn’t involve am. Otherwise, if F | L ⊢1 am, unit propagation has falsified some αi; every subsequent propagation step from F | L that uses (ām ∨ βj) can use (αi ∨ βj) in a propagation step from G | L. A similar argument applies when F | L ⊢1 ām.

(Incidentally, the elimination of an auxiliary variable also preserves “honesty.”)

(b) False. Let F = (x_1 ∨ x_2 ∨ a_1) ∧ (x_1∨ x_2∨ ā_1), L=x¯1 or x¯2.

427. Suppose n = 3m, and let f be the symmetric function [νx<m or νx>2m]. The prime clauses of f are the N=(nm,m,m)∼3n+3/2(2πn) ORs of m positive literals and m negative literals. There are N′=(nm−1,m,m+1)=mm+1N ways to specify that xi_1 = … = xim = 1 and xim+1 = … = xi2m–1 = 0; and this partial assignment implies that xj = 1 for j ∉ {i_1,...,i_2m–1}. Therefore at least one of the m + 1 clauses (x¯i1 ∨…∨x¯im∨xim+1∨…∨xi2m−1∨xj) must be present in any set of prime clauses that forces f. By symmetry, any such set must include at least N′/m prime clauses.

On the other hand, f is characterized by O(n^2) forcing clauses (see answer 436).

428. (a) (y ∨ zj_1∨…∨ zjd) for 1 ≤ j ≤ n; (x¯ij∨z¯ik∨z¯jk) for 1 ≤ i < j ≤ n, 1 ≤ k ≤ d.

(b) Imagine a circuit with 2N(N + 1) gates glt, one for each literal l of Gnd and for each 0 ≤ t ≤ N, meaning that literal l is known to be true after t rounds of unit propagation, if we start with given values of the xij variables only. Thus we set gl_0 ← 1 if l = xij and xij is true, or if l=x¯ij and xij is false; otherwise gl_0 ← 0. And
gl(t+1)←glt∨∨{gl1t ∧…∧ gl¯kt| (l∨l1 ∨…∨ lk)∈Gnd},for 0≤t<N.

Given values of the xij, the literal y is implied if and only if the graph has no d-coloring; and at most N rounds make progress. Thus there’s a monotone chain for gyN=f¯nd.

[This exercise was suggested by S. Buss and R. Williams in 2014, based on a similar construction by M. Gwynne and O. Kullmann.]

429. Let Σk be the sum of the assigned x’s in leaves descended from node k. Unit propagation will force bjk←1 for 1 ≤ j ≤ Σk, moving from leaves toward the root. Then it will force bjk←0 for j = Σk + 1, moving downwards from the root, because r = Σ_2 +Σ_3 and because (21) starts this process when k = 2 or 3.

430. Imagine boundary conditions as in answer 26, and assume that xj_1, ... , xjr have been assigned 1, where j_1 < … < jr. Unit propagation forces sjk+1−kk←1 for 1 ≤ k ≤ r; then it forces sjk−kk←0 for r ≥ k ≥ 1. So unassigned x’s are forced to zero.

431. Equivalently x1+…+xm+y¯1+…+y¯n≤n; so we can use (18)–(19)or (20)–(21).

432. The clauses of answer 404(b) can be shown to be forcing. But not those of 404(a) when a > 1; for example, if a = 2 and we assume x¯2, unit propagation doesn’t yield y^2.

433. Yes. Imagine, for example, the partial assignment x = 1∗∗∗10∗∗1, y = 10∗00∗1∗∗. Then y_3 must be 1; otherwise we’d have 10010001 ≤ x ≤ y ≤ 100001111. In this situation unit propagation from the clauses that correspond to 1 ≤ 〈a_101〉, a1≤〈a2x¯20〉, a2≤〈a3x¯3y3〉, a3≤〈a4x¯40〉, a_4 ≤ a_500 forces a_1 = 1, a_2 = 1, a_4 = 0, a_3 = 0, y_3 = 1.

In general if a given partial assignment is consistent with x ≤ y, we must have x↓ ≤ y↑, where x↓ and y↑ are obtained from x and y by changing all unassigned variables to 0 and 1, respectively. If that partial assignment forces some yj to a particular value, the value must be 1; and we must in fact have x↓ > y′↑, where y′ is like y but with yj = 0 instead of yj = ∗. If xj ≠ 1, unit propagation will force a_1 = … = aj–1 = 1, ak = … = aj = 0, yj = 1, for some k ≥ j.

Similar remarks apply when xi is forced, because x≤y⇔y¯≤x¯.

434. (a) Clearly pk is equivalent to x¯1∧…∧x¯k, qk is equivalent to x¯k∧…∧x¯n, and rk implies that a run of exactly l 1s begins at xk.

(b) When l = 1, if xk = 1 unit propagation will imply p¯j for j ≥ k and q¯j for j ≤ k, hence r¯j for j ≠ k; then rk is forced, making xj = 0 for all j ≠ k. Conversely, xj = 0 forces r¯j; if this holds for all j ≠ k, then rk is forced, making xk = 1.

But when l = 2 and n = 3, the clauses fail to force x_2 = 1 by unit propagation. They also fail to force x_1 = 0 when we have l = 2, n = 4, and x_3 = 1.

435. The following construction with O(nl) clauses is satisfactory when l is small: Begin with the clauses for pk and qk (but not rk) in exercise 434(a); include also (x¯k∨pk−l) for l < k ≤ n, and (x¯k∨qk+l) for 1 ≤ k ≤ n – l. Append (p¯k−l∨q¯k+l∨xk) for 1 ≤ k ≤ n, omitting p¯j for j < 1 and omitting q¯j for j > n. Finally, append

(xk∨x¯k+1∨xk+d)for 0≤k<n  and  1<d<1,(∗)

omitting xj when j < 1 or j > n.

To reduce to O(n log l) clauses, suppose 2e+1 < l ≤ 2e+2, where e ≥ 0. The clauses (∗) can be replaced by (x¯k∨y¯k(e)∨z¯k(e)) for 1 ≤ k ≤ n, if x¯k−d implies yk(e) for 1 ≤ d ≤ ⌊l/2⌋ and x¯k+d implies zk(e) for 1 ≤ d ≤ ⌈l/2⌉. And to achieve the latter, we introduce clauses (y¯k(t)∨yk(t+1)),(y¯k−2t(t)∨yk(t+1)),(z¯k(t)∨zk(t+1)),(z¯k+2t(t)∨zk(t+1)), (xk−1∨yk(0)),(xk+2e−1−⌊l/2⌋∨yk(0)),(xk+1∨zk(0)),(xk−2e+1+⌈l/2⌉∨zk(0)), for 1 ≤ k ≤ n and 0 ≤ t < e, always omitting xj or y¯j or z¯j when j < 1 or j > n.

436. Let the variables qk for 0 ≤ k ≤ n and q ∈ Q represent the sequence of states, and let tkaq represent a transition when 1 ≤ k ≤ n and when T contains a triple of the form (q′,a,q). The clauses, F , are the following, for 1 ≤ k ≤ n: (i) (t¯kaq∨xka)∧(t¯kaq∨qk), where xk0 denotes x¯k and xk1 denotes xk; (ii) (q¯k−1∨∨{tkaq′|(q,a,q′)∈T}), for q ∈ Q; (iii) (q¯k∨∨{tkaq|(q′,a,q)∈T}); (iv) (x¯ka∨∨{tkaq| (q′,a,q)∈T}); (v) (t¯kaq′∨∨{qk−1|(q,a,q′)∈T}), for a ∈ A, q′ ∈ Q. And (vi) (q¯0) for q ∈ Q\I, (q¯n) for q ∈ Q\O.

It is clear that if F⊢1 x¯ka no string x_1 ...xn ∈ L can have xk = a. Conversely, assume that F⊬1 x¯ka, and in particular that F ⊬1 ∊. To prove the forcing property, we want to show that some string of L has xk = a. It will be convenient to say that a literal l is ‘n.f.’ (not falsified) if F⊬1 l¯; thus xka is assumed to be n.f.

By (iv), there’s a (q′,a,q) ∈ T such that tkaq is n.f. Hence qk is n.f., by (i). If k = n we have q ∈ O by (vi); otherwise some t(k+1)bq′ is n.f., by (ii), hence xk+1b is n.f. Moreover, (v) tells us that there’s (q″,a,q) ∈ T with qk−1″ n.f. If k = 1 we have q″ ∈ I; otherwise some t(k–1)cq″ is n.f., by (iii), and xk−1c is n.f. Continuing this line of reasoning yields x_1 ...xn ∈ L with xk = a (and with xk+1 = b if k < n, xk–1 = c if k > 1).

The same proof holds even if we add unit clauses to F that assign values to one or more of the x’s. Hence F is forcing. [See F. Bacchus, LNCS 4741 (2007), 133–147.]

For example, the language L_2 of exercise 434 yields 20n + 4 clauses with 8n +3 auxiliary variables: F=∧k=1n((t¯k00∨x¯k)∧(t¯k00∨0k)∧(t¯k11∨xk)∧(t¯k11∨1k) ∧ (t¯k12∨xk)∧(t¯k12∨2k)∧(t¯k02∨x¯k)∧(t¯k02∨2k)(0¯k−1∨tk00∨tk11)∧(1¯k−1∨tk12) ∧ (2¯k−1∨tk02)∧(0¯k∨tk00)∧(1¯k∨tk11)∧(2¯k∨tk02∨tk12)∧(xk∨tk00∨tk02)∧(x¯k∨tk11∨tk12) ∧ (t¯k00∨0k−1)∧(t¯k11∨0k−1)∧(t¯k12∨1k−1)∧(t¯k02∨2k−1))∧(1¯0)∧(2¯0)∧(0¯n)∧(1¯n).

The clauses produced by this general-purpose construction can often be significantly simplified by preprocessing to eliminate auxiliary variables. (See exercise 426.)

437. Each variable xk now becomes a set of |A| variables xka for a ∈ A, with clauses like (15) and (17) to ensure that exactly one value is assigned. The same construction is then valid, with the same proof, if we simply replace ‘xka’ by ‘xka’ throughout. (Notice that unit propagation will often derive partial information such as x¯ka, meaning that xk ≠ a, although the precise value of xk may not be known.)

438. Let l≤j = l_1 + ··· + lj. Exercise 436 does the job via the following automaton: Q = {0, 1,...,l≤t + t − 1}, I = {0}, O = {l≤t + t − 1}; T = {(l≤j + j, 0, l≤j + j) | 0 ≤ j < t} ∪ {(l≤j + j + k, 1, l≤j + j + k + 1) | 0 ≤ j < t, 0 ≤ k < lj+1} ∪ {(l≤j + j − 1, 0, l≤j + j − [j = t]) | 1 ≤ j ≤ t}.

439. We obviously want the clauses (x¯j∨x¯j+1) for 1 ≤ j < n; and we can use, say, (18) and (19) with r = t, to force 0s whenever the number of 1s reaches t. The difficult part is to force 1s from partial patterns of 0s; for example, if n = 9 and t = 4, we can conclude that x_4 = x_6 = 1 as soon as we know that x_3 = x_7 = 0.

An interesting modification of (18) and (19) turns out to work beautifully, namely with the clauses (t¯jk∨tj+1k) for 1 ≤ j < 2t − 1 and 1 ≤ k ≤ n − 2t + 1, together with (x2j+k−1∨t¯2j−1k∨t2j−1k+2) for 1 ≤ j ≤ t and 0 ≤ k ≤ n − 2t + 1, omitting t¯2j−10 and t2j−1n−2t+2.

440. It’s convenient to introduce (n+12)|N| variables Pik for all P ∈ N and for 1 ≤ i ≤ k ≤ n, as well as (n+13)|N|2 variables QRijk for Q, R ∈ N and for 1 ≤ i < j ≤ k ≤ n, although almost all of them will be eliminated by unit propagation. The clauses are: (i) (QR¯ijk∨Qi(j−1))∧(QR¯ijk∨Rjk); (ii) (P¯kk∨∨{xka | P→a∈U}); (iii) (P¯ik∨∨{QRijk | i<j≤k,P→QR∈W}), if i < k; (iv) (x¯ka∨∨{Pkk | P→a∈U}); (v) (P¯ik∨∨{PRi(k+1)l | k<l≤n,R∈N}∨∨{QRhik | 1≤h<i,Q∈N}), if i > 1 or k < n; (vi) (QR¯ijk∨∨{Pik | P→QR∈W}); (vii) (P¯1n) for P ∈ N \ S.

The forcing property is proved by extending the argument in answer 436: Assume that xka is n.f.; then some Pkk with P → a is also n.f. Whenever Pik is n.f. with i > 1 or k < n, some PRi(k+1)l or QPhik is n.f.; hence some “larger”Pil′ or Phk′ is also n.f. And if P_1n is n.f., we have P ∈ S.

Furthermore we can go “downward”: Whenever Pik is n.f. with i < k, there’s QRijk such that Qi(j−1) and Rjk are n.f.; on the other hand if Pkk is n.f., there’s a ∈ A such that xka is n.f. Our assumption that xka is n.f. has therefore shown the existence of x_1 ...xn ∈ L with xk = a.

[See C.-G. Quimper and T. Walsh, LNCS 4741 (2007), 590–604].

441. See O. Bailleux, Y. Boufkhad, and O. Roussel, LNCS 5584 (2009), 181–194.

442. (a) F | Lq−=F | l1 | . . . | lq−1 | l¯q contains ϵ if and only if F |l_1 | ... | lq−1 contains ϵ or the unit clause (lq).

(b) If F ⊬1 l and F | l¯ ⊢1 ϵ, the failed literal elimination technique will reduce F to F | l and continue looking for further reductions. Thus we have F ⊢2 l if and only if unit propagation plus failed literal elimination will deduce either ϵ or l.

(c) Use induction on k; both statements are obvious when k = 0. Suppose we have F ⊢k+1 l¯ via l1,…,lp=l¯, with F | Lq− ⊢k ϵ for 1 ≤ q ≤ p. If p > 1 we have F | l | Lq− ⊢k ϵ for 1 ≤ q < p; it follows that F | l ⊢k+1 lp−1 and F | l ⊢k+1 l¯p−1. If p = 1 we have F | l | Lq− ⊢k ϵ for 1 ≤ q < p; it follows that F | l ⊢k+1 lp−1 and F ⁢|  l ⊢k+1 l¯p−1. If p = 1 we have F | l ⊢k ϵ. Hence F | l ⊢k+1 ϵ in both cases.

Now we want to prove that F | l ⊢k+1 ϵ and F ⊢k+2 ϵ , given F ⊢k+1 l′ and F ⁢⊢k+1 l¯′. If F ⁢ | Lq− ⊢k ϵ for 1 ≤ q ≤ p, with lp = l′, we know that F ⁢ | Lq− ⊢k+1 ϵ. Furthermore we can assume that F ⊬k+1 l¯; hence l≠l¯q for 1 ≤ q ≤ p, and l ≠ lp. If l = lq for some q < p, then F | l | Lr− ⊢k ϵ for 1 ≤ r < q and F | Lr− ⊢k ϵ for q < r ≤ p; otherwise F | l| Lq− ⊢k ϵ for 1 ≤ q ≤ p. In both cases F | l ⊢k+1 l′ and F ⊢k+2 l′. Essentially the same proof shows that F | l⊢k+1 l¯′ and F ⊢k+2 l¯′.

(d) True, by the last relation in part (c).

(e) If all clauses of F have more than k literals, Lk(F ) is empty; hence L_0(R′) = L_1(R′) = L_2(R′) = ∅. But Lk(R′)={1¯,2,4} for k ≥ 3; for example, R′ ⊢3 1¯ because R′ | 1 ⊢2 ϵ, because R′ | 1 ⊢2 3 and R′ | 1⊢2 3¯.

(f) Unit propagation can be done in O(N) steps if N is the total length of all clauses; this handles the case k = 1.

For k ≥ 2, procedure Pk(F ) calls Pk−1(F | x_1), Pk−1 (F | x¯1), Pk−1(F | x_2), etc., until either finding Pk−1 (F | l¯)={ϵ} or trying both literals for each variable of F . In the latter case, Pk returns F . In the former case, if Pk−1(F | l) is also {ϵ}, Pk returns {ϵ}; otherwise it returns Pk(F | l). The set Lk contains all literals for which we’ve reduced F to F | l, unless Pk(F ) = { ϵ}. (In the latter case, every literal is in Lk.)

To justify this procedure we must verify that the order of testing literals doesn’t matter. If F | l¯ ⊢k ϵ and F | l¯′ ⊢k ϵ, we have F | l | l¯′⊢k ϵ and F | l′ | l¯⊢k ϵ by (c); hence Pk(F | l) = Pk(F | l | l′) = Pk(F | l′ | l) = Pk(F | l′).

[See O. Kullmann, Annals of Math. and Artificial Intell. 40 (2004), 303–352.]

443. (a) If F | L ⊢ ϵ then F | L ⊢ l for all literals l; so if F ∈ PCk we have F | L ⊢k l and F | L ⊢k l¯ and F | L ⊢k ϵ , proving that PCk ⊆ UCk.

Suppose F ∈ UCk and F | L ⊢ l. Then F | L | l¯ ⊢ϵ, and we have F | L | l¯ ⊢k ϵ. Consequently F | L ⊢k+1 l, proving that UCk ⊆ PCk+1.

The satisfiable clause sets ∅, {1}, {1, 1¯2}, {12, 1¯2}, {12, 1¯2, 12¯, 1¯2¯3}, {123, 1¯23, 12¯3, 1¯2¯3}, {123, 1¯23, 12¯3, 1¯2¯3, 123¯, 1¯23¯, 12¯3¯, 1¯2¯3¯4}, ... , show that PCk ≠ UCk ≠ PCk+1.

(b) F ∈ PC_0 if and only if F = ∅ or ϵ ∈ F . (This can be proved by induction on the number of variables in F , because ϵ ∉ F implies that F has no unit clauses.)

(c) If F has only one clause, it is in UC_0. More interesting examples are {12¯, 1¯2}; {1234, 1¯2¯3¯4¯}; {1234¯, 123¯4, 12¯34, 1¯234}; {12, 1¯2¯, 345¯, 3¯4¯5}; etc. In general, F is in UC_0 if and only if it contains all of its prime clauses.

(d) True, by induction on n: If F | L ⊢ l then F | L | l¯ ⊢ϵ, and F | L | l¯ has ≤ n − 1 variables; so F | L | l¯∈PCn−1⊆UCn−1. Hence we have F | L | l¯⊢n−1 ϵ and F | L ⊢n l.

(e) False, by the examples in (c).

(f) R′ ∈ UC_2 \ PC_2. For example, we have R′ | 1 ⊢2 2 and R′ | 1 ⊢2 2¯.

[See M. Gwynne and O. Kullmann, arXiv:1406.7398 [cs.CC] (2014), 67 pages.]

444. (a) Complementing a variable doesn’t affect the algorithm’s behavior, so we can assume that F consists of unrenamed Horn clauses. Then all clauses of F will be Horn clauses of length ≥ 2 whenever step E2 is reached. Such clauses are always satisfiable, by setting all remaining variables false; so step E3 cannot find both F ⊢1 l and F⊢1 l¯.

(b) For example, {12, 2¯3, 12¯3¯, 1¯23}.

(c) Every unsatisfiable F recognized by SLUR must be in UC_1. Conversely, if F ∈ UC_1, we can prove that F is satisfiable and in UC_1 whenever step E2 is reached.

[Essentially the same argument proves that a generalized algorithm, which uses ⊢k instead of ⊢1 in steps E1 and E3, always classifies F if and only if F ∈ UCk. See M. Gwynne and O. Kullmann, Journal of Automated Reasoning 52 (2014), 31–65.]

(d) If step E3 interleaves unit propagation on F | l with unit propagation on F | l¯, stopping when either branch is complete and ϵ was not detected in the other, the running time is proportional to the number of cells used to store F , using data structures like those of Algorithm L. (This is an unpublished idea of Klaus Truemper.)

[SLUR is due to J. S. Schlipf, F. S. Annexstein, J. V. Franco, and R. P. Swami-nathan, Information Processing Letters 54 (1995), 133–137.]

445. (a) Since the lexicographic constraints (169) are forcing, a succinct certificate is (x¯1m,x¯2m, . . . , x¯(m−1)m, x¯2(m−1), x¯3(m−1), x¯3(m−1), ... , x¯(m−1)(m−1), x¯3(m−2), x¯4(m−2), ... , x¯(m−1)(m−2), x¯(m−1)2, ∅). The first m − 1 steps can be replaced by ‘x_0m’.

(b) (x¯(m−1)1,x¯(m−2)2,. . . , x¯1(m−1),∅).

(c) (x_01, x_12, ... , x(m−2)(m−1), ∅).

446. Z(m, n) − 1, because a 4-cycle corresponds to a quad.

447. For general m and n we can add the m3¯n3¯/3! constraints (x¯ij ∨x¯i′j ∨ x¯i′j′ ∨ x¯i′′j′ ∨ x¯i′′j′′ ∨ x¯ij′′) to (184), for 1 ≤ i < i′ < i″ ≤ m and distinct {j, j′,j″} ⊆ {1,...,n}. The 19-edge graph illustrated here works when m = n = 8; and Algorithm C finds girth ≥ 8 unsatisfiable with 20 edges, after only 400 megamems of calculation (using lexicographic row/column symmetry).

448. Each pair of points can occur together in at most one line. If the lines contain respectively l_1, ... , ln points, we therefore have (l12)+⋅⋅⋅+(ln2)≤(m2)=3n. A Steiner triple system achieves equality, with l_1 = ··· = ln = 3. Since (l−12)+(l′+12)<(l2)+(l′2) when l ≥ l′ + 2, we can’t have l_1 + ··· + ln > 3n. Thus Z(m, n) = 3n +1.

[If m is even and (m2)=3n, we can’t cover all the pairs with triples, because no point can be in more than (m − 2)/2 triples. Daniel Horsley proved in 2015 that Z(m, n) = 3n + ⌊1 − m/14⌋ in such cases.]

449. It’s wise to try first for symmetric solutions with xij = xji, roughly halving the number of variables; then the matrices below are found quickly. Such solutions are impossible when n = 9, 12, 13 (and also when n = 15 and 16 if we insist on five 1s in the top row). The case n = 13 corresponds to the projective plane of order 3; indeed, a projective plane of order q is equivalent to a maximum quad-free matrix with m = n = q^2 + q + 1 and Z(n, n) = (q + 1)n + 1.

450. To prove the hint, add the unary clause (x¯15) to the others; this problem is rapidly found to be unsatisfiable, hence no line has more than 4 points. On the other hand, a line with fewer than 3 points is impossible because Z(9, 10) = 32. The same arguments show that every point belongs to either 3 or 4 lines. Thus exactly four lines contain four points, and exactly four points lie on such lines.

If p ∈ l and l is a 4-point line, every other line containing p must contain 2 of the remaining 6 points. And the four 4-point lines contain at least 4×4−(42)=10 points altogether. Hence, pigeonwise, we see that each of the four 4-point lines contains exactly one of the four 4-line points.

Now we may call the 4-line points {a, b, c, d}, and the 4-point lines {A, B, C, D}. The other points may be called {ab, ac, ad, bc, bd, cd}, with A = {a, ab, ac, ad}, B = {b, ab, bc, bd}, C = {c, ac, bc, cd}, D = {d, ad, bd, cd}. The other lines can be called {AB,AC,AD,BC,BD,CD}; and we have AB = {a, b, cd}, AC = {a, c, ad}, etc.

451. One of the colors can be placed uniquely, by the previous exercise. So we’re left with the simple problem of two-coloring the remaining 66 squares and avoiding both 0-quads and 1-quads. That problem is unsatisfiable with ∑ xij odd. The author then constructed a 33 + 33 + 33 solution by hand, using the fact that each color class must be unable to use the deleted square. [See M. Beresin, E. Levine, and J. Winn, The College Mathematics Journal 20 (1989), 106–114 and the cover; J. L. Lewis, JRM 28 (1997), 266–273.]

452. Any such solution must have exactly 81 cells of each color, because R. Nowakowski proved in 1978 that Z(18, 18) = 82. The solution exhibited here was found by B. Steinbach and C. Posthoff [Multiple-Valued Logic and Soft Computing 21 (2013), 609–625], exploiting 90° rotational symmetry.

453. (a) If R ⊆ {1,...,m} and C ⊆ {1,...,n}, let V (R, C)= {ui | i ∈ R}∪{vj | j ∈ C}. If X is decomposable, there’s no path from a vertex in V (R, C) to a vertex not in V (R, C); hence the graph isn’t connected. Conversely, if the graph isn’t connected, let V (R, C)be one of its connected components. Then 0 < |R|+|C| < m+n, and we’ve decomposed X.

(b) False in general, unless every row and column of X′ contains a positive element. Otherwise, clearly true by the definition of lexicographic order.

(c) True: A direct sum is certainly decomposable. Conversely, let X be decomposable via R and C. We may assume that 1 ∈ R or 1 ∈ C; otherwise we could replace R by {1,...,m} \ R and C by {1,...,n} \ C. Let i ≥ 1 and j ≥ 1 be minimal such that i ∉ R and j ∉ C. Then xi′j = 0 for 1 ≤ i′ < i and xij′ = 0 for 1 ≤ j′ < j. The lexicographic constraints now force xi′j′ = 0 for 1 ≤ i′ < i, j′ ≥ j; also for i′ ≥ i, 1 ≤ j′ < j. Consequently X = X′ ⊕ X″, where X′ is (i − 1) × (j − 1) and X″ is (m + 1 − i) × (n + 1 − j). (Degenerate cases where i = 1 or j = 1 or i = m + 1 or j = n + 1 need to be considered, but they work fine. This result allows us to “read off” the block decomposition of a lexicographically ordered matrix.)

Reference: A. Mader and O. Mutzbauer, Ars Combinatoria 61 (2001), 81–95.

454. We have f(x) ≤ f(xτ) ≤ f(xτ τ) ≤ ··· ≤ f(xτk) ≤ ··· ; eventually xτk = x.

455. (a) Yes, because C only causes 1001 and 1011 to be nonsolutions. (b) No, because F might have been satisfied only by 0011. (c) Yes as in (a), although (187) might no longer be an endomorphism of F ∧ C as it was in that case. (d) Yes; if 0110 is a solution, so are 0101 and 1010. [Of course this exercise is highly artificial: We’re unlikely to know that a weird mapping such as (187) is an endomorphism of F unless we know a lot more about the set of solutions.]

456. Only (1 + 2 · 7)(1 + 2)(1 + 8) = 405, out of 65536 possibilities (about 0.06%).

457. We have min_0≤k≤16(kk1616−k) = 6^616^10 ≈ 51.3 × 10^15. For general n, the minimum occurs when k = 2n/e+O(1); and it is 22n(n−x) where x = 1/(e ln 2)+O(2−n) < 1.

458. The operation of assigning values to each variable of an autarky, so that all clauses containing those variables are satisfied, while leaving all other variables unchanged, is an endomorphism. (For example, consider the operation that makes a pure literal true.)

459. sweep(Xij) = −∞ when i = 0 or j = 0. And for 1 ≤ i ≤ m and 1 ≤ j ≤ n we have sweep(Xij) = max(xij + sweep(X(i−1)(j−1)), sweep(X(i−1)j), sweep(Xi(j−1))).

[Let the 1s in the matrix be xi1j_1 , ... , xirjr, with 1 ≤ i_1 ≤ ··· ≤ ir ≤ m and with jq+1 < jq when iq+1 = iq. Richard Stanley has observed (unpublished) that sweep(X) is the number of rows that occur when the Robinson–Schensted–Knuth algorithm is used to insert the sequence n − j_1, ... , n − jr into an initially empty tableau.]

460. We introduce auxiliary variables sijt that will become true if sweep(Xij) > t. They are implicitly true when t < 0, false when t = k. The clauses are as follows, for 1 ≤ i ≤ m, 1 ≤ j ≤ n, and 0 ≤ t ≤ min(i − 1, j − 1, k): (s¯(i−1)jt ∨ sijt), if i > 1 and t < k; (s¯i(j−1)t ∨ sijt), if j > 1 and t < k; and (x¯ij ∨ s¯(i−1)(j−1)t−1∨ sijt). Omit s¯0(j−1)t−1 and s¯(i−1)0t−1 and s¯(i−1)(j−1)−1 and sijk from that last clause, if present.

461. ∧i=1m−1 ∧j=1n−1 (xij ∨ c¯(i−1)j ∨ cij) ∧∧i=1m ∧j=1n−1 (c¯(i−1)j ∨ x¯ij ∨ xi(j+1)), omitting c¯0j. These clauses take care of τ_1; interchange i ↔ j, m ↔ n for τ_2.

462. Let X˜ij denote the last m + 1 − i rows and the last n + 1 − j columns of X; and let tij = sweep(X(i−1)(j−1)) + sweep(X˜(i+1)(j+1)). For τ_1 we must prove 1 + ti(j+1) ≤ k, given that 1 + tij ≤ k. It’s true because sweep(X(i−1)j) = sweep(X(i−1)(j−1)) when column j begins with i − 1 zeros, and we have sweep(X˜(i+1)(j+2))≤sweep(X˜(i+1)(j+1)).

Let X′ = Xτ_3 have the associated sweep sums tij′. We must prove that tij′≤k and 1+t(i+1)(j+1)′≤k, if 1 + tij ≤ k, 1 + ti(j+1) ≤ k, 1 + t(i+1)j ≤ k, and t(i+1)(j+1) ≤ k. The key point is that sweep(Xij′) = max(sweep(X(i−1)j), sweep(Xi(j−1))), since xij′=0. Also sweep(X˜(i+1)(j+1)′)≤1+sweep(X˜(i+2)(j+1)).

(Notice that τ_1 and τ_2 might actually decrease the sweep, but τ_3 preserves it.)

463. If row i + 1 is entirely zero but row i isn’t, τ_2 will apply. Therefore the all-zero rows occur at the top. And by τ_1, the first nonzero row has all its 1s at the right.

Suppose rows 1 through i have r_1, ... , ri 1s, all at the right, with ri > 0. Then r_1 ≤ ··· ≤ ri, by τ_2. If i < n we can increase i to i + 1, since we can’t have x(i+1)j > x(i+1)(j+1) when j ≤ n−ri, by τ_1; and we can’t have it when j > n−ri, by τ_3.

Thus all the 1s are clustered at the right and the bottom, like the diagram of a partition but rotated 180°; and the sweep is the size of its “Durfee square” (see Fig. 91 in Section 7.2.1.4). Hence the maximum number of 1s, given sweep k, is k(m + n − k).

[Under the partial ordering (i, j) ≺ (i′, j′) when i < i′ and j < j′, binary matrices of sweep ≤ k correspond to sets of cells with all chains of length ≤ k. Significant lattice and matroid properties of such “Sperner k-families” have been studied by C. Greene and D. J. Kleitman, J. Combinatorial Theory A20 (1976), 41–68.]

464. By answer 462, τ_1 can be strengthened to τ1″, which sets xi(j+1) ← 1 but leaves xij = 1. Similarly, τ2″ can be strengthened to τ″2. These endomorphisms preserve the sweep but increase the weight, so they can’t apply to a matrix of maximum weight. [One can prove, in fact, that max-weight binary matrices of sweep k are precisely equivalent to k disjoint shortest paths from the leftmost cells in row m to the rightmost cells in row 1. Hence every integer matrix of sweep k is the sum of k matrices of sweep 1.]

465. If not, there’s a cycle x_0 → x_1 → ··· → xp = x_0 of length p > 1, where xiτuvi ↦ xi+1. Let uv be the largest of {uv_1,...,uvp−1}. Then none of the other τ ’s in the cycle can change the status of edge uv. But that edge must change status at least twice.

(See also the more general result in Theorem 7.2.2.1S.)

466. Notice first that v_11 must be true, if m ≥ 2. Otherwise h_11, v_21, h_22, v_32, ... would successively be forced by unit propagation, until reaching a contradiction at the edge of the board. And v_31 must also be true, if m ≥ 4, by a similar argument. Thus the entire first column must be filled with verticals, except the bottom row when m is odd.

Then we can show that the remainder of row 1 is filled with horizontals, except for the rightmost column when n is even. And so on.

The unique solution when m and n are both even uses vij if and only if i + j is even and 1 ≤ j ≤ min(i, m − i, n/2), or i + j is odd and vi(n+1−j) is used. When m is odd, add a row of horizontals below the (m − 1) × n solution. When n is odd, remove the rightmost column of verticals in the m × (n + 1) solution.

467. The 8 × 7 covering is the reflection of the 7 × 8 covering (shown here) about its southwest-to-northeast diagonal. Both solutions are unique.

468. (a) Typical running times with Algorithm C for sizes 6 × 6, 8 × 8, ... , 16 × 16 are somewhat improved: 39 Kμ, 368 Kμ, 4.3Mμ, 48Mμ, 626 Mμ, 8Gμ.

(b) Now they’re even better, but still growing exponentially: 30 Kμ, 289 Kμ, 2.3Mμ, 22Mμ, 276 Mμ, 1.7Gμ.

469. For instance (v_11), (v_31), (v_51), (h_12), (h_14), (v_22), (v_42), (h_23), (v_33), ϵ.

470. There can’t be a cycle x_0 → x_1 → ··· → xp = x_0 of length p > 1, because the largest vertex whose mate is changed always gets smaller and smaller mates.

471. We must pair 2n with 1, then 2n − 1 with 2, ... , then n + 1 with n.

472. We can number the vertices from 1 to mn in such a way that every 4-cycle switches as desired. For example, we can make (i, j) < (i, j + 1) ⇔ (i, j) < (i +1,j) ⇔ (i, j) mod 4 ∉ {(0, 0), (0, 1), (1, 1), (1, 2), (2, 2), (2, 3), (3, 3), (3, 0)}. One such numbering in the 4 × 4 case is shown here.

473. For every even-length cycle v_0 −−− v_1 −−−···−−− v_2r−1 −−− v_0 with v_0 = max vi and v_1 > v_2r−1, assert (v0v1¯ ∨ v1v2 ∨ v2v3¯ ∨⋅⋅⋅∨ v2r−1 v0).

474. (a) (2n) · (2n − 2) · ... · 2 = 2nn!. (b) (173¯)(1¯7¯3)(252¯5¯)(44¯)(6)(6¯).

(c) Using 0, 1, ... , f for the 4-tuples 0000, 0001, ... , 1111, we must have f(0) = f(9) = f(5); f(2) = f(b) = f(7); f(4) = f(8) = f(d); and f(6) = f(a) = f(f); in other words, the truth table of f must have the form abcdeagceagcfehg, where a, b, c, d, e, f, g, h ∉ {0, 1}. So there are 2^8 f’s.

(d) Change ‘=’ to ‘≠’ in (c). There are no such truth tables, because (191) contains odd cycles; all cycles of an antisymmetry must have even length.

(e) The 128 binary 7-tuples are partitioned into sixteen “orbits” {x, xσ, xσ^2,... }, with eight of size 12 and eight of size 4. For example, one of the 4s is {0011010, 0010110, 0111110, 0110010}; one of the 12s is {0000000, 0011101,..., 1111000}. Hence there are 2^16 functions with this symmetry, and 2^16 others with this antisymmetry.

475. (a) 2n+1 n!. (There are 2n+1 n!/a, if f has a automorphisms+antiautomorphisms.)

(b) (xz¯)(x¯z), because (surprisingly) (x∨y) ∧ (x ⊕ z)=(z¯∨y)∧(z¯ ⊕ x¯).

(c) In general if σ is any permutation having a cycle of length l, and if p is a prime divisor of l, some power of σ will have a cycle of length p. (Repeatedly raise σ to the qth power for all primes q ≠ p, until all cycle lengths are powers of p. Then, if the longest remaining cycle has length pe, compute the pe−1 st power.)

(d) Suppose f(x_1, x_2, x_3) has the symmetry (x1x¯2x3)(x¯1x2x¯3). Then f(0, 0, 0) = f(1, 1, 0) = f(0, 1, 1), f(1, 1, 1) = f(0, 0, 1) = f(1, 0, 0), so (x1x¯2)(x¯1x2) is a symmetry.

(e) A similar argument shows that (ux)(vw)(u¯x¯)(v¯w¯) is a symmetry.

(f) If σ is an antisymmetry of f, then σ^2 is a symmetry. If f has a nontrivial symmetry, it has a symmetry of prime order p, by (c). And if p ≠ 2, it has one of order 2, by (d) and (e), unless n > 5.

(g) Let f(x_1,...,x_6) = 1 only when x_1 ...x_6 ∈ {001000, 001001, 001011, 010000, 010010, 010110, 100000, 100100, 100101}. (Another interesting example, for n = 7, has f = 1 ⇔ x_1 ...x_7 is a cyclic shift of 0000001, 0001101, or 0011101; 21 symmetries.)

476. We want clauses that specify r-step chains in n variables, having a single output xn+r. For 0 < t < t′ < 2n, introduce new variables Δtt′ = x(n+r)t ⊕ x(n+r)t′. (See (24).) Then for each signed involution σ, not the identity, we want a clause that says “σ is not a symmetry of f,” namely (∨{Δtt′ | t < t′ and t′ = tσ}). (Here t is considered to be the same as its binary representation (t_1 ...tn)2, as in exercise 477.)

Also, if σ has no fixed points — this is true if and only if σ takes xi↦x¯i for at least one i — we have further things to do: In case (b), we want a clause that says “σ is not an antisymmetry,” namely (∨{Δ¯tt′ | t<t′ and t′=tσ}). But in case (a), we need further variables aj for 1 ≤ j ≤ T, where T is the number of signed involutions that are fixedpoint-free. We append the clause (a_1 ∨···∨ aT), and also (a¯j∨Δtt′) for all t < t′ such that t′ = tσ when σ corresponds to index j. Those clauses say, “there’s at least one signed involution that is an antisymmetry.”

There are no solutions when n ≤ 3. Answers for (a) are (((x_1 ⊕ x_2) ∨ x_3) ∧ x_4) ⊕ x_1 and (((x¯1 ⊕ x2) ∧ x3) ⊕ x4) ∧x5)⊕x1; in both cases the signed involution (11¯)(22¯) is obviously an antisymmetry. Answers for (b) are ((x_1 ⊕ x_2) ∨ x_3) ∧ (x_4 ∨ x_1) and (((x_1 ∧ x_2) ⊕ x_3) ∧ x_4) ⊕ (x_5 ∨ x_1). [Is there a simple formula that works for all n?]

477. Use the following variables for 1 ≤ h ≤ m, n < i ≤ n + r, and 0 < t < 2n: xit = (tth bit of truth table for xi); ghi = [gh = xi ]; sijk = [xi = xj ◦i xk], for 1 ≤ j < k < i; fipq = ◦i(p, q) for 0 ≤ p, q ≤ 1, p + q > 0. (We don’t need fi_00, because every operation in a normal chain takes (0, 0) ↦ 0.) The main clauses for truth table computations are (s¯ijk∨(xit ⊕ a)∨(xjt ⊕ b)∨(xkt ⊕ c)∨(fibc ⊕ a¯)), for 0 ≤ a, b, c ≤ 1 and 1 ≤ j < k < i.

Simplifications arise in special cases: For example, if b = c = 0, the clause is omitted if a = 0, and the term fi_00 is omitted if a = 1. Furthermore if t = (t_1 ... tn)2, and if j ≤ n, the (nonexistent) variable xjt actually has the known value tj; again we omit either the whole clause or the term (xjt ⊕ b), depending on b and t. For example, there usually are eight main clauses that involve sijk; but there’s only one that involves si_12 when t < 2n−2, namely (s¯i12∨x¯i1), because the truth tables for x_1 and x_2 begin with 2n−2 0s. (All such simplifications would be done by a preprocessor if we had defined additional variables fi_00 and xjt, and fixed their values with unit clauses.)

There also are more mundane clauses, namely (g¯hi∨x¯it) or (g¯hi∨xit) according as gh(t_1,...,tn) = 0 or 1, to fix the outputs; also (∨i=n+1n+r ghi) and (∨k=1i−1 ∨j=1k−1 sijk), to ensure that each output appears in the chain and that each step has two operands.

Additional clauses are optional, but they greatly shrink the space of possibilities: (∨k=1m gki∨∨i′=i+1n+r ∨j=1i−1 si′ji∨∨i′=i+1n+r∨j=i+1i′−1 si′ij) ensures that step i is used at least once; (s¯ijk∨s¯i′ji) and (s¯ijk∨s¯i′ki) for i < i′ ≤ n + r avoid reapplying an operand.

Finally, we can rule out trivial binary operations with the clauses (fi_01 ∨ fi_10 ∨ fi_11), (fi01∨f¯i10∨f¯i11), (f¯i01∨fi10∨f¯i11). (But beware: These clauses, for n < i ≤ n + r, will make it impossible to compute the trivial function g_1 = 0 in fewer than three steps!)

Further clauses such as (s¯ijk∨fi10∨x¯it∨xjt) are true, but unhelpful in practice.

478. We can insist that the (j, k) pairs in steps n + 1, ... , n + r appear in colexicographic order; for example, a chain step like x_8 = x_4 ⊕ x_5 need never follow x_7 = x_2 ∧ x_6. The clauses, for n < i < n + r, are (s¯ijk∨s¯(i+1)j′k′) if 1 ≤ j′ < j < k = k′ < i or if 1 ≤ j < k and 1 ≤ j′ < k′ < k < i. (If (j, k) = (j′, k′), we could insist further that fi_01 fi_10 fi_11 is lexicographically less than f(i+1)01 f(i+1)10 f(i+1)11. But the author didn’t go that far.)

Furthermore, if p<q and if each output function is unchanged when xp is swapped with xq, we can insist that xp is used before xq as an operand. Those clauses are

(s¯ijq∨∨n<i′<i ∨i≤j′<k′<i′ [j′=p or k′=p] si′j′k′) whenever j≠p.

For example, when answer 477 is applied to the full-adder problem, it yields Mr clauses in Nr variables, where (M_4, M_5) = (942, 1662) and (N_4, N_5) = (82, 115). The symmetry-breaking strategy above, with (p, q) = (1, 2) and (2, 3), raises the number of clauses to Mr′, where (M4′,M5′)=(1025,1860). Algorithm C reported ‘unsat’ after (1015, 291) kilomems using (M4,M4′) clauses; ‘sat’ after (250, 268) kilomems using (M5,M5′). With larger problems, such symmetry breakers give significant speedup when proving unsatisfiability, but they’re often a handicap in satisfiable instances.

479. (a) Using the notation of the previous answer, we have (M8,M8′,N8)=(14439,17273,384) and (M9,M9′,N9)=(19719,24233,471). The running times for the ‘sat’ cases with M_9 and M9′ clauses were respectively (16, 645, 1259) and (66, 341, 1789) megamems — these stats are the (min, median, max) of nine runs with different random seeds. The ‘unsat’ cases with M_8 and M8′ were dramatically different: (655631, 861577, 952218) and (8858, 10908, 13171). Thus s(4) = 9 in 7.1.2–(28) is optimum.

(b) But s(5) = 12 is not optimum, despite the beauty of 7.1.2–(29)! The M_11 = 76321 clauses in N_11 = 957 variables are ‘sat’ in 680 Gμ, yielding an amazing chain:

x6=x1 ⊕ x2,x10=x6∨x7,x7=x1 ⊕ x3,x11=x4 ⊕ x9,x14=x¯8 ∧ x11,x8=x4 ⊕ x5,x12=x9 ⊕ x10,z1=x15=x10 ⊕ x14,x9=x3 ⊕ x6,z0=x13=x5 ⊕ x11,z2=x16=x12 ∧ x¯15.

And (M10′,N10)=(68859,815) turns out to be ‘unsat’ in 1773 gigamems; this can be reduced to 309 gigamems by appending the unit clause (g3(15)), since C(S_4,5) = 10.

Hence we can evaluate x_1 +···+ x_7 in only 5 + 11 + 2 + 1 = 19 steps, by computing (u_1u_0)2 = x_5+x_6+x_7, (v_2v_1z_0)2 = x_1+x_2+x_3+x_4+u_0, (w_2z_1)2 = u_1+v_1, z_2 = v_2 ⊕ w_2.

(c) The solver finds an elegant 8-step solution for (M_8, N_8) = (6068, 276) in 6 Mμ:

x4=x1∨x2,x6=x3 ⊕ x4x8=x3 ⊕ x5,S1=x10=x6∧x8,x5=x1 ⊕ x2,S¯0=x7=x3∨x4,S3=x9=x¯6 ∧x8,S2=x11=x7 ⊕ x8.

The corresponding (M7′, N7)=(5016,217) problem is ‘unsat’ in 97 Mμ.

(d) The total cost of evaluating the S’s independently is 3 + 7 + 6 + 7 + 3 = 26, using the optimum computations of Fig. 9 in Section 7.1.2. Therefore the author was surprised to discover a 9-step chain for S_1, S_2, and S_3, using the footprint heuristic:

x5=x1 ⊕ x2,x8=x5 ⊕ x7,S2=x11=x¯8 ∧ x9,x6=x1 ⊕ x3,x9=x6 ∨ x7,S3=x12=x8 ∧ x¯10,x7=x3 ⊕ x4,x10=x2 ⊕ x9,S1=x13=x8 ∧ x10.

This chain can solve problem (d) in 13 steps; but SAT technology does it in 12(!):

x5=x1 ⊕ x2,x9=x6∨ x7,S1=x13=x8 ∧ x10,x6=x1⊕ x3,x10=x2⊕ x9,S4=x14=x1 ∧ x¯11,x7x8==x3 ⊕ x4,x5 ⊕ x7,x11S3=x12==x5∨ x9,x8 ∧ x¯10,S¯0S2==x15=x4 ∨ x11,x16=x¯8 ∧ x11.

The nonexistence of an 11-step solution can be proved via Algorithm C by a long computation (11034 gigamems), during which 99,999,379 clauses are learned(!).

(e) This solution (found in 342 Gμ) matches the lower bound in exercise 7.1.2–80:

x7=x1 ⊕ x2,x11=x4 ⊕ x10,x8=x3 ⊕ x4,x12=x5 ⊕ x10,x9=x1 ⊕ x5,x13=x8 ∨x11,x10=x6 ⊕ x8,x14=x7 ⊕ x12,x15=x¯9 ∧ x12,x16=x13 ⊕ x15,x17=x14 ∧ x16.

(f) This solution (found in 7471 Gμ) also matches that lower bound:

x7=x1 ∧ x2,x11=x5 ⊕ x6,x15=x8 ⊕ x13,x8=x1 ⊕ x2,x12=x4 ⊕ x11,x16=x10 ⊕ x14,x9=x3 ⊕ x4,x13=x9 ⊕ x11,x17=x7 ⊕ x16,x10=x5 ∧ x6,x14=x9 ∨ x12,x18=x15 ∨ x17,

Here x_18 is the normal function S¯0,4=S1,2,3,5,6. We beat exercise 7.1.2–28 by one step.

(g) A solution in t(3) = 12 steps is found almost instantaneously (120 megamems); but 11 steps are too few (‘unsat’ in 301 gigamems).

480. (a) Let x_1x_2x_3x_4 = xlxrylyr. The truth tables for zl and zr are 0011010010001000 and 01∗∗1∗00∗011∗011, where the ∗s (“don’t-cares”) are handled by simply omitting the corresponding clauses (g¯hi∨±xit) in answer 477.

Less than 1 gigamem of computation proves that a six-step circuit is ‘unsat’. Here’s a seven-stepper, found in just 30 Mμ: x_5 = x_2 ⊕ x_3, x_6 = x_3 ∨ x_4, x_8 = x_1 ⊕ x_6, x_7 = x_1 ∨ x_5, x_9 = x_6 ⊕ x_7, zl = x_10 = x_7 ∧ x_8, zr = x_11 = x_3 ⊕ x_9. (See exercise 7.1.2–60 for a six-step solution that is based on a different encoding.)

(b) Now we have the truth tables zl = 00110100010010000100100010000011, zr = 01∗∗1∗001∗00∗0111∗00∗011∗01101∗∗, if x_4x_5 = ylyr. One of many 9-step solutions is found in 6.9 gigamems: x_6 = x_1⊕ x_2, x_7 = x_2 ⊕ x_5, x_8 = x_4 ⊕ x_6, x9=x¯4∧x7, x_10 = x_1 ⊕ x_9, x_11 = x_8 ∨ x_9, x_12 = x_3 ⊕ x_10, zr = x_13 = x_3 ⊕ x_11, zl=x14=x11 ∧ x¯12.

The corresponding clauses for only 8 steps are proved ‘unsat’ after 190 Gμ of work. (Incidentally, the encoding of exercise 7.1.2–60 does not have a 9-step solution.)

(c) Let cn be the minimum cost of computing the representation zlzr of (x_1 + ··· + xn) mod 3. Then (c_1, c_2, c_3, c_4) = (0, 2, 5, 7), and cn−3 ≤ cn + 9. Hence cn ≤ 3n − 4 for all n ≥ 2. [This result is due to A. Kojevnikov, A. S. Kulikov, and G. Yaroslavtsev, whose paper in LNCS 5584 (2009), 32–44, also inspired exercises 477–479.]

Conjecture: For n ≥ 3 and 0 ≤ a ≤ 2, the minimum cost of evaluating the (single) function [(x_1 + ··· + xn) mod 3 = a] is 3n − 5 − [(n + a) mod 3 = 0]. (It’s true for n ≤ 5. Here’s a 12-step computation when n = 6 and a = 0, found in 2014 by Armin Biere: x_7 = x_1 ⊕ x_2, x_8 = x_3 ⊕ x_4, x_9 = x_1 ⊕ x_5, x_10 = x_3 ⊕ x_5, x_11 = x_2 ⊕ x_6, x_12 = x_8 ⊕ x_9, x_13 = x_8 ∨ x_10, x_14 = x_7 ⊕ x_13, x15=x¯12 ∧ x13, x16=x¯11 ∧ x14, x_17 = x_11 ⊕ x_15, S¯0,3,6=x18=x16∨x17. The case n = 6 and a ≠ 0, which lies tantalizingly close to the limits of today’s solvers, is still unknown. What is C(S_1,4(x_1,...,x_6))?)

481. (a) Since z ⊕ z′ = 〈x_1x_2x_3〉 and z′ = x_1 ⊕ x_2 ⊕ x_3, this circuit is called a “modified full adder.” It costs one less than a normal full adder, since z′ = (x_1 ⊕ x_2) ⊕ x_3 and z = (x_1 ⊕ x_2) ∨ (x_1 ⊕ x_3). (And it’s the special case u = 0 of the more general situation in exercise 7.1.2–28.) Part (b) describes a “modified double full adder.”

(b) The function z_2 has 20 don’t-cares, so there are many eight-step solutions (although 7 are impossible); for example, x_6 = x_1 ⊕ x_5, x_7 = x_2 ⊕ x_5, z_3 = x_8 = x_3 ⊕ x_6, x_9 = x_4 ⊕ x_6, x_10 = x_1 ∨ x_7, x11=x¯3∧x9, z_2 = x_12 = x_6 ⊕ x_11, z_1 = x_13 = x_10 ⊕ x_11.

(c) Letting y_2k−1y_2k = ⟦x_2k−1x_2k⟧, it suffices to show that the binary representation of Σn = ν⟦y_1y_2⟧ + ··· + ν⟦y_2n−1y_2n⟧ + y_2n+1 can be computed in at most 8n steps. Four steps are enough when n = 1. Otherwise, letting c_0 = y_2n+1, we can compute z’s bits with ν⟦y_4k−3y_4k−2⟧ + ν⟦y_4k−1y_4k⟧ + ck−1 = 2ν⟦z_2k−1z_2k⟧ + ck for 1 ≤ k ≤ ⌊n/2⌋. Then Σn = 2(ν⟦z_1z_2⟧ + ··· + ν⟦zn−1 zn⟧) + cn/2 if n is even, Σn = 2(ν⟦z_1 z_2⟧ +···+ ν⟦zn−2zn−1⟧+zn) + c′ if n is odd, where ν⟦y_2n−1y_2n⟧ + c ⌊n/2⌋ = 2zn +c′, at a cost of 4n in both cases. The remaining sum costs at most 8⌊n/2⌋ by induction. [See E. Demenkov, A. Kojevnikov, A. S. Kulikov, and G. Yaroslavtsev, Information Processing Letters 110 (2010), 264–267.]

482. (a) Σj=1k (2yi−1) is odd when k is odd, and it’s ±1 when k = 1.

(b) Adapting Sinz’s cardinality clauses as in exercises 29 and 30, we only need the auxiliary variables aj=sjj−1, bj=sjj, and cj=sjj+1, because sjj+2=0 and sj+2j=1. The clauses are then (b¯j∨aj+1)∧(c¯j∨bj+1)∧(bj∨c¯j)∧(aj+1∨b¯j+1), for 1 ≤ j < t/2 − 1; and (y¯2j−2∨aj)∧(y¯2j−1∨a¯j∨bj)∧(y¯2j∨b¯j∨cj) ∧ (y¯2j+1∨c¯j) ∧(y2j−2∨c¯j−1) ∧ (y2j−1 ∨ cj−1∨ b¯j) ∧ (y2j∨bj∨a¯j+1)∧(y2j+1∨aj+1) for 1 ≤ j < t/2, omitting ā_1, c_0, and the two clauses that contain y_0.

(c) Use the construction in (b) with yj = xjd for 1 ≤ d ≤ n/3 and independent auxiliary variables aj,d, bj,d, cj,d. Also, assuming that n ≥ 720, break symmetry by asserting the unit clause (x_720). (That’s much better than simply asserting (x_1).)

This problem was shown to be satisfiable if and only if n < 1161 by B. Konev and A. Lisitsa [Artificial Intelligence 224 (2015), 103–118], thereby establishing the case C = 2 of a well-known conjecture by Paul Erdős [Michigan Math. J. 4 (1957), 291–300, Problem 9]. Algorithm C can prove unsatisfiability for n = 1161 in less than 600 gigamems, using the parameters of exercise 512.

483. Using a direct encoding as in (15), with vjk meaning that vj has color k, we can generate the clauses (v¯jk) for 1 ≤ j < k ≤ d and (v¯j(k+1)∨∨i=kj−1 vik) for 2 ≤ k < j ≤ n. A similar but slightly simpler scheme works with the order encoding, when vjk means that vj has color > k. [See Ramani, Markov, Sakallah, and Aloul, Journal of Artificial Intelligence Research 26 (2006), 289–322. The vertices might be ordered in such a way that degree(v_1) ≥ ··· ≥ degree(vn), for example.]

It’s not difficult to color the Mycielski graph Mc with c colors (which is the minimum), without any symmetry breaking. For example, the 191-vertex graph M_12 leads to 2,446,271 clauses in 36852 variables (total length 4.9 million); yet 12-color solutions are found by Algorithms C, W, and L respectively in 2.6, 523, and 12200 megamems. The symmetry breaking clauses actually would retard that calculation, because those clauses are much longer. On the other hand, when we try to succeed with only c − 1 colors, those clauses are extremely helpful: The runtime needed by Algorithm C to show that M_6 isn’t 5-colorable goes down from 124 Gμ to 32 Mμ! Furthermore, Algorithm L does better here: Its runtime for that problem goes down from 7.5 Gμ to 28 Mμ.

484. (a) A type (iii) move will work if and only if v_1 −−− v_4, v_2 −−− v_4, v_2 −−− v_3.

(b) For 0 ≤ t < n − 1 we have the clause (∨k=1n−t−1 qt,k ∨∨l=1n−t−3 st,l), as well as the following for 1 ≤ i < j < n − t, 1 ≤ k < n − t, 1 ≤ l < n − t − 2: (q¯t,k∨xt,k,k+1); (q¯t,k∨x¯t+1,i,j∨xt,i′,j′); (s¯t,l∨xt,l,l+3); (s¯t,l∨x¯t+1,i,j∨xt,i′′,j′′); here i′ = i + [i ≥ k], j′ = j + [j ≥ k], and {i″, j″} are the min and max of {i + [i ≥ l + 3] + 3[i = l], j +[j ≥ l + 3] + 3[j = l]}. Finally there’s a unit clause (x¯0,i,j) for all 1 ≤ i < j ≤ n with vi —vj.

(These clauses essentially compute [G is quenchable], which is a monotone Boolean function of the (n2) elements above the diagonal in the adjacency matrix of G. The prime implicants of this function correspond to certain spanning trees, of which there are respectively 1, 1, 2, 6, 28, 164, 1137, ... when n = 1, 2, 3, 4, 5, 6, 7, ... .)

485. Let t′ = t + 1. Instances of commutativity are: (qt,k,qt′, k′) ↔ (qt, k + 1, qt′, k) if k < k′; (st,l′, st′, l′) ↔ (st′,l′+1, st′, l) if l + 2 < l′; (qt,k′, st′, l′) ↔ (st,l′+1, qt′, k) if k < l′; (st,l, qt′, k′) ↔ (qt,k′+1, st′, l) if l + 2 < k′; (st,l, st′, l) ↔ (qt,l+3, st′,l). These can be broken by appending the clauses (q¯t,k′+1∨q¯t′,k), (s¯t,l′+1∨s¯t′,l), ... , (q¯t,l+3∨s¯t′,l).

Endomorphisms are also present in the two cases (qt,k,qt′,k) ↔ (qt,k+1, qt′,k) and (st,k+1, qt′,k) ↔ (qt,k+1,st′,k), provided that both pairs of transitions are legal. These are exploited by the clauses (q¯t,k+1∨q¯t′,k∨x¯t,k,k+1) and (q¯t,k+1∨s¯t′,k∨x¯t,k+1,k+4).

486. This game is a special case of graph quenching, so we can use the previous two exercises. Algorithm C finds a solution after about 1.2 gigamems, without the symmetry-breaking clauses; this time goes down to roughly 85 megamems when those clauses are added. Similarly, the corresponding 17-card problem after A♣ × J♣ is found to be unsatisfiable, after 15 Gμ without and 400 Mμ with. (A♣ × × 10♣ fails too.)

Those SAT problems have respectively (1242, 20392, 60905), (1242, 22614, 65590), (1057, 15994, 47740), (1057, 17804, 51571) combinations of (variables, clauses, cells), and they are not handled easily by Algorithms A, B, D, or L. In one solution both q_0,11 and s_0,7 are true, thus providing two ways to win(!), when followed by q_1,15, s_2,13, q_3,12, s_4,10, s_5,7, q_6,7, s_7,5, q_8,5, s_9,4, q_10,5, s_11,3, q_12,3, s_13,1, s_14,1, q_15,1, q_16,1.

Notes: This mildly addictive game is an interesting way to waste time in case you ever get lost with a pack of cards on a desert island. If you succeed in reducing the original 18 piles to a single pile, you can continue by dealing 17 more cards and trying to reduce the new 18 piles. And if you succeed also at that, you have 17 more cards for a third try, since 52 = 18 + 17 + 17. Three consecutive wins is a Grand Slam.

In a study of ten thousand random deals, just 4432 turned out to be winnable. Computer times (with symmetry breaking) varied wildly, from 1014 Kμ to 37 Gμ in the satisfiable cases (median 220 Mμ) and from 46 Kμ to 36 Gμ in the others (median 848 Mμ). The most difficult winnable and unwinnable deals in this set were respectively

9♠ 7♣ 3♣ K◊ 7♠ 3♡ 2◊ 8♣ 6♡ J◊ 8♠ 2♡ 6♠ 4◊ 5♠ 4♡ 10◊ Q♠ and A♡ Q♡ 2◊ 9◊ 7♣ 7◊ 8♡ K♣ 3◊ 10♣ 3♣ 3♠ Q♠ 8♣ 2♣ K♠ 6◊ 5♣ .

Students in Stanford’s graduate problem seminar investigated this game in 1989 [see K. A. Ross and D. E. Knuth, Report STAN-CS-89-1269 (Stanford Univ., 1989), Problem 1]. Ross posed an interesting question, still unsolved: Is there a sequence of (say) nine “poison cards,” such that all games starting with those cards are lost?

The classic game Idle Year is also known by many other names, including Tower of Babel, Tower of London, Accordion, Methuselah, and Skip Two. Albert H. Morehead and Geoffrey Mott-Smith, in The Complete Book of Solitaire and Patience Games (1949), 61, suggested that moves shouldn’t be too greedy.

487. Every queen in a set of eight must attack at least 14 vacant cells. Thus |∂S| gets its minimum value 8 × 14 = 112 when the queens occupy the top row. Solutions to the 8 queens problem, when queens are independent, all have |∂S| ≤ 176. The maximum |∂S| is 184, achieved symmetrically for example in Fig. A–11(a). (This problem is not at all suitable for SAT solvers, because the graph has 728 edges. The best way to proceed is to run through all (648) possibilities with the revolving-door Gray code (Algorithm 7.2.1.3R), because incremental changes to |∂S| are easy to compute when a queen is deleted or inserted. The total time by that method is only 601 gigamems.)

The maximum of |∂outS| is obviously 64 − 8 = 56. The minimum, which corresponds to Turton’s question, is 45; it can be achieved symmetrically as in Fig. A–11(b), leaving 64 − 8 − 45 = 11 cells unattacked (shown as black queens). In this case SAT solvers win: The revolving-door method needs 953 gigamems, but SAT methods show the impossibility of 44 after only 2.2 Gμ of work. With symmetry reduction as in the following exercise, this goes down to 900 Mμ although there are 789 variables and 4234 clauses. [Bernd Schwarzkopf, in Die Schwalbe 76 (August 1982), 531, computed all solutions of minimum |∂outS|, given |S|, for n × n boards with n ≤ 8. Extensions of Turton’s problem to larger n have been surveyed by B. Lemaire and P. Vitushinskiy in two articles, written in 2011 and accessible from www.ffjm.org. Optimum solutions for n > 16 are conjectured but not yet known.]

All sets S of eight queens trivially have |∂inS| = 8.

488. Let variables wij and bij represent the presence of white or black queens on cell (i, j), with clauses (w¯ij∨b¯i′j′) when (i, j) = (i′, j′) or (i, j) −−− (i′, j′). Also, if each army is to have at least r queens, add clauses based on (20) and (21) to ensure that ∑ wij ≥ r and ∑ bij ≥ r. Optionally, add clauses based on Theorem E to ensure that k of the w variables for the top row are lexicographically greater than or equal to the corresponding k variables in fifteen symmetrical variants. (For instance, if k = 3, we might require w_11w_12w_13 ≥ b_1nb_2nb_3n, thus partially breaking the symmetries.)

Fig. A–11. Optimum queen placements of various kinds.

The maximum army sizes for 3 ≤ n ≤ 13 are known to be (1, 2, 4, 5, 7, 9, 12, 14, 17, 21, 24); see OEIS sequence A250000. An extra black queen can actually be included in the cases n = 3, 4, 6, 8, 10, 11, and 13. Solutions appear in Fig. A–11; the construction shown in Fig. A–11(d) generalizes to armies of 2q(q + 1) queens whenever n = 4q + 1, while the one in part (c) belongs to another family of constructions that achieve the higher asymptotic density 748n2.

When n = 8 and r = 9, Algorithm C typically finds a solution in about 10 megamems (k = 0), or about 30 megamems (k = 3); but with r = 10 it typically proves unsatisfiability in about 1800 Mμ (k = 0) or 850 Mμ (k = 3) or 550 Mμ (k = 4) or 600 Mμ (k = 5). Thus the symmetry breaking constraints are helpful for unsatisfiability in this case, but not for the easier satisfiability problem. On the other hand, the extra constraints do turn out to be helpful for both the satisfiable and unsatisfiable variants when n is larger. The “sweet spot” turns out to be k = 6 when n = 10 and n = 11; unsatisfiability was proved in those cases, with r = 15 and r = 18, after about 185 Gμ and 3500 Gμ, respectively. [B. M. Smith, K. E. Petrie, and I. P. Gent obtained similar results using CSP methods in LNCS 3011 (2004), 271–286.]

(This problem was posed by S. Ainley in his Mathematical Puzzles (1977), problem C1. He mentioned solutions for n ≤ 30 that have never yet been beaten, although he obtained them by hand. See also Martin Gardner, Math Horizons 7, 2 (November 1999), 2–16, for generalizations to coexisting armies of sizes r and s. D. M. Kane has proved, among other things, that the maximum value of s, if r = 3q^2 + 3q + 1, is asymptotically n^2 − (6q + 3)n + O(1) [arXiv:1703.04538 [math.CO] (2017), 19 pages].)

489. T_0 = 1, T_1 = 2, Tn = 2Tn−1 +(2n − 2)Tn−2 (see Eq. 5.1.4–(40)). The generating function ∑n Tnzn/n! and the asymptotic value are given in exercise 5.1.4–31.

490. Yes. For example, using the signed permutation 4¯132¯, we’re allowed to assume that some solution satisfies x¯4x1x3x¯2≤x¯4′x1′x3′x¯2′ for every endomorphism — because the solution with lexicographically smallest x¯4x1x3x¯2 has this property. Notice that the signed permutation 1¯2¯ ... n¯ converts ‘≤’ to ‘≥’.

491. Let σ be the permutation (1234 1¯ 2¯ 3¯ 4¯). Then σ^4 = (1 1¯)(2 2¯)(3 3¯)(4 4¯); and by Theorem E we need only search for solutions that satisfy x1x2x3x4≤x¯1x¯2x¯3x¯4. We’re therefore allowed to append the clause (x¯1) without affecting satisfiability.

(We actually are allowed to assert that x_1 = x_2 = x_4 = 0, because 0000 and 0010 are the lex-leaders of the two 8-cycles when σ is written as a permutation of states.)

In general if an automorphism σ is a permutation of literals having a cycle that contains both v and v¯, for some variable v, we can simplify the problem by assigning a fixed value to v and then by restricting consideration to automorphisms that don’t change v. (See the discussion of Sims tables in Section 7.2.1.2.)

492. Suppose x_1 ...xn satisfies all clauses of F ; we want to prove that (x1 . . . xn)τ=x1′ . . . xn′ also satisfies them all. And that’s easy: If (l_1 ∨···∨ lk) is a clause, we have l1′=l1τ,. . . , ln′=ln τ; and we know that (l_1τ ∨···∨ lkτ ) is true because it’s subsumed by a clause of F . [See S. Szeider, Discrete Applied Math. 130 (2003), 351–365.]

493. Using the global ordering p_1 ...p_9 = 543219876 and Corollary E, we can add clauses to assert that x_5 = 0 and x_4x_3x_2x_1 ≤ x_6x_7x_8x_9. A contradiction quickly follows, even if we stipulate only the weaker relation x_4 ≤ x_6, because that forces x_6 = 1.

494. Exercise 475(d) shows that (uv)(u¯v¯) is a symmetry of the underlying Boolean function, although not necessarily of the clauses F . [This observation is due to Aloul, Ramani, Markov, and Sakallah in the cited paper.] The other symmetries allow us to assert (i) (x¯i∨xj)∧(x¯j∨x¯k), (ii) (x¯i∨x¯j)∧(x¯j∨x¯k), (iii) (x¯i∨x¯j)∧(x¯j∨xk).

495. Suppose, for example, that m = 3 and n = 4. The variables can then be called 11, 12, 13, 14, 21, ... , 34; and we can give them the global ordering 11, 12, 21, 13, 22, 31, 14, 23, 32, 24, 33, 34. To assert that 21 22 23 24 ≤ 31 32 33 34, we use the involution that swaps rows 2 and 3; this involution is (21 31)(22 32)(23 33)(24 34) when expressed in form (192) with signs suppressed. Similarly we can assert that 12 22 13 ≤ 13 23 33 because of the involution (12 13)(22 23)(32 33) that swaps columns 2 and 3. The same argument works for any adjacent rows or columns. And we can replace ‘≤’ by ‘≥’, by complementing all variables.

For general m and n, consider any global ordering for which xij precedes or equals xi j when 1 ≤ i ≤ i′ ≤ m and 1 ≤ j ≤ j′ ≤ n. The operation of swapping adjacent rows makes the global lexicographic order increase if and only if it makes the upper row increase lexicographically; and the same holds for columns.

[See Ilya Shlyakhter, Discrete Applied Mathematics 155 (2007), 1539–1548.]

496. No; that reasoning would “prove” that m pigeons cannot fit into m holes. The fallacy is that his orderings on rows and columns aren’t simultaneously consistent with a single global ordering, as in the previous exercise.

497. A BDD with 71,719 nodes makes it easy to calculate the total, 818,230,288,201, as well as the generating function 1 + z +3z^2 +8z^3 +25z^4 + ··· + 21472125415z^24 + 31108610146z^25 + ··· + 10268721131z^39 + 6152836518z^40 + ··· + 24z^60 + 8z^61 + 3z^62 + z^63 + z^64. (The relatively small coefficients of z^39 and z^40 help account for the fact that ≥ was chosen in (185)–(186); problems with sparse solutions tend to favor ≥.)

[Pólya’s theorem in Section 7.2.3 shows that exactly 14,685,630,688 inequivalent matrices exist; compare this to 2^64 ≈ 1.8447 × 10^19 without any symmetry reduction.]

498. Consider the global ordering x_01, x_11, ... , xm_1; x_12, x_22, ... , xm_2, x_02; x_23, x_33, ... , xm_3, x_03, x_13; ... ; x(m−1)m, xmm, x_0m, ... , x(m−2)m. There’s a column symmetry that fixes all elements preceding x(j−1)j and takes x(j−1)j ↦ x(j−1)k.

499. No. The unusual global ordering in answer 498 is not consistent with ordinary lexicographic row or column ordering. [Nor can the analogous clauses (xii∨x¯ij) for 1 ≤ i ≤ m and i < j ≤ n be appended to (185) and (186). No quad-free matrix for m = n = 4 and r = 9 satisfies all those constraints simultaneously.]

500. If F_0 has a solution, then it has a solution for which l is true. But (F_0 ∪ F_1) | l might be unsolvable. (For example, let F0=(x¯1∨x2)∧(x¯2∨x1), which has the symmetry 1¯2¯; so we can take S=(x¯1),l=x¯1. Combine that with F_1 = (x_1).)

501. Let xij denote a queen in cell (i, j), for 1 ≤ i ≤ m and 1 ≤ j ≤ n. Also let rij = [xi_1 + ··· + xij ≥ 1] and r′ij = [xi_1 + ··· + xi(j+1) ≥ 2], for 1 ≤ i ≤ m and 1 ≤ j < n. Using (18) and (19) we can easily construct about 8mn clauses that define the r’s in terms of the x’s and also ensure that xi_1 + ··· + xin ≤ 2. Thus ri(n−1)′=[xi1+⋅⋅⋅+xin=2]; call this condition ri.

Similar conditions cj, ad, and bd are readily established for column j, and for the diagonals with i+j = d + 1 or i−j = d−n, for 1 ≤ i ≤ m,1 ≤ j ≤ n, and 1 ≤ d < m+n. Then condition (ii) corresponds to the mn clauses (xij ∨ ri ∨ cj ∨ ai+j−1 ∨ bi−j+n).

Finally we have clauses from (20) and (21) to ensure that ∑ xij ≤ r.

When m = n, the lower bound r ≥ n − [n mod 4 = 3] has been established by A. S. Cooper, O. Pikhurko, J. R. Schmitt, and G. S. Warrington [AMM 121 (2014), 213–221], who also used backtracking to show that r ≥ 12 on an 11 × 11 board. SAT methods, with symmetry breaking, yield that result much more quickly (after about 9 teramems of computation); but this problem, like the tomography problem of Fig. 79, is best solved by integer programming techniques when m and n are large.

If we call the upper left corner white, solutions with m = n = r −1 and all queens on white squares appear to exist for all n > 2, and they are found almost instantly. However, no general pattern is apparent. In fact, when n is odd it appears possible to insist that the queens all appear in odd-numbered rows and in odd-numbered columns.

Here are examples of optimum placements on smallish boards. The solutions for 8 × 9, 8 × 10, 8 × 13, 10 × 10, and 12 × 12 also work for sizes 8 × 8, 9 × 10, 8 × 12, 9 × 9, and 11 × 11, respectively.

This placement of ten queens on a 10 × 10 board can be described by the “magic sequence” (a_1,...,a_5) = (1, 3, 7, 5, 9), because the queens appear in positions (ai, ai+1) and (ai+1, ai) for 1 ≤ i < n/2 as well as in (a_1,a_1) and (an/2, an/2). The magic sequences (1, 3, 9, 13, 15, 5, 11, 7, 17) and (9, 3, 1, 19, 5, 11, 15, 25, 7, 21, 23, 13, 17) likewise describe optimum placements for n = 18 and 26. No other magic sequences are known; none exist when n = 34.

502. For each j, construct independent cardinality constraints for the relation x1(j)+⋅⋅⋅+xn(j)≤rj, using say (20) and (21), where xk(j)=(sjk?x¯k: xk).

503. The Hamming distance d(x, y) = ν(x ⊕ y) between binary vectors of length n satisfies d(x,y)+d(x¯,y)=n. Thus there is no x with d(x, sj) ≥ rj + 1 for all j if and only if there is no x with d(x¯,sj)≤n−1−rj for all j. [See M. Karpovsky, IEEE Transactions IT-27 (1981), 462–472.]

504. (a) Assume that n ≥ 4. For strings of length 2n we have d(z,w)+d(z,w¯=2n); hence d(z, w) ≤ n and d(z,w¯)≤n if and only if d(z,w)=d(z,w¯)=n. Every string z with z_2k−1 ≠ z_2k for 1 ≤ k ≤ n satisfies d(z, wj) = n for 1 ≤ j ≤ n. Conversely, if d(z, wj) = d(z, wk) = n and 1 ≤ j < k ≤ n, then z_2j−1 + z_2j = z_2k−1 + z_2k. Thus if z_2j−1 = z_2j for some j we have z = 00 ... 0 or 11 ... 1, contradicting d(z, w_1) = n.

(b) For each string x^=x¯1x1x¯2x2 . . . x¯nxn that satisfies part (a) we have d(x^,y)=2l¯1+2l¯2+2l¯3+n−3, which is ≤ n + 1 if and only if (l_1 ∨ l_2 ∨ l_3) is satisfied.

(c) Let sj = wj and rj = n for 1 ≤ j ≤ 2n; let s_2n+k = yk and r_2n+k = n +1 for 1 ≤ k ≤ m, where yk is the string in (b) for the kth clause of F . This system has a closest string x^=x¯1x1x¯2x2 . . . x¯nxn if and only if x_1 ...xn satisfies every clause. [A similar construction in which all strings have length 2n + 1 and all rj are equal to n + 1 is obtained if we append the bit [n < j ≤ 2n] to each sj. See M. Frances and A. Litman, Theory of Computing Systems 30 (1997), 113–119.]

(d) Boilerplate 11000000, 00110000, 00001100, 00000011, 00111111, 11001111, 11110011, 00000011, at distance ≤ 4; for the clauses, 01011000, 00010110, 01000101, 10010001, 10100100, 00101001, 10001010, and possibly 01100010, at distance ≤ 5.

505. (For k = 0, 1, ... , n − 1 one can set j to a uniform integer in [0..k] and INX[k + 1] ← j; also if j = k set VAR[k] ← k + 1, otherwise i ← VAR[j], VAR[k] ← i, INX[i] ← k, VAR[j] ← k + 1.) With nine random seeds, typical runtimes for D3 are (1241, 873, 206, 15, 748, 1641, 1079, 485, 3321) Mμ. They’re much less variable for the unsatisfiable K0, namely (1327, 1349, 1334, 1330, 1349, 1322, 1336, 1330, 1317) Mμ; and even for the satisfiable W2: (172, 192, 171, 174, 194, 172, 172, 170, 171) Mμ.

506. (a) Almost true: That sum is the total number of clauses of length ≥ 2, because every such clause of length k contributes 1/(k2) to the weights of (k2) edges.

(b) Each of the 12^2 − 2 = 142 cells of the mutilated 12 × 12 board contributes one positive clause (v_1 ∨···∨ vk) and (k2) negative clauses (v¯i∨v¯j), when that cell can be covered by k potential dominoes {v_1,...,vk}. So the weight between u and v is 2, 4/3, or 7/6 when dominoes u and v overlap in a cell that can be covered in 2, 3, or 4 ways. Exactly 6 cells can be covered in just 2 ways (and exactly 10^2 in 4 ways).

(The largest edge weights in all of Fig. 95 are 37/6, between 20 pairs of vertices in K6. At the other extreme, 95106 of the 213064 edges in X3 have the tiny weight 1/8646, and 200904 of them have weight at most twice that much.)

507. Consider, for example, the clauses (u∨t¯),(v∨t¯),(u¯∨v¯∨t),(u∨t¯′),(v∨t¯′), (u¯∨v¯∨t′) from (24). Looking ahead from t = 1 yields the windfall (t¯∨t′), and looking ahead from t′ = 1 yields (t¯′∨t). Henceforth Algorithm L knows that t equals t′.

508. According to (194), the purging parameters were Δp = 1000 and δp = 500; thus we have learned approximately 1000k+500(k2) clauses when doing the kth purging phase. After 1000L clauses this works out to be ≈(16L+9−3)/2 phases, which is ≈ 34.5 when L = 323. (And the actual number was indeed 34.)

509. One remedy for overfitting is to select training examples at random. In this case such randomness is already inherent, because of the different seeds used while training.

510. (a) From Fig. 96 or Fig. 97 or Table 7 we know that T1 < T2 < L6 in the median rankings; thus T2 obscures L6 and T1.

(b) Similarly, L8 < M3 < Q2 < X6 < F2 < X4 < X5; X6 obscures L8 and X4.

(c) X7 obscures K0, K2, and (indirectly) A2, because K2 obscures K0 and A2.

511. (a) Nine random runs finished in only (4.9, 5.0, 5.1, 5.1, 5.2, 5.2, 5.3, 5.4, 5.5) Mμ(!).

(b) Nine random runs now each were aborted after a teramem of trials. (No theoretical explanation for this discrepancy, or for the wildness of P4 in Fig. 97, is known.)

(c) (0.2,..., 0.5,..., 3.2) Mμ without; (0.3,..., 0.5,..., 0.7) Mμ with.

512. A training run with ParamILS in 2015 suggested the parameters
α=0.7,ρ=0.998,ϱ=0.99995,Δp=100000,δp=2000,  τ=10,w=1,p=0,P=0.05,ψ=0.16667,(*)

which produce the excellent results in Fig. A–12.

513. After training on rand (3, 1062, 250, 314159), ParamILS chose the values α = 3.5 and Θ = 20.0 in (195), together with distinctly different values that favor double lookahead, namely β = .9995, Y = 32. [The untuned values α = 3.3, β = .9985, Θ = 25.0, and Y = 8 had been used by the author when preparing exercise 173.]

Fig. A–12. Running times for Algorithm C, with and without special parameter tuning.

514. ParamILS suggests p = .85 and N = 5000n; that gives a median time ≈ 690 Mμ. (But those parameters give horrifically bad results on most other problems.)

515. Use variables Sijk meaning that cell (i, j) in the solution holds k, and Zij meaning that cell (i, j) is blank in the puzzle. The 729 S variables are constrained by 4×81×(1+(92))=11,988 clauses like (13). From condition (i), we need only 41 variables Zij. Condition (ii) calls for 15 clauses such as (Z_11 ∨· · ·∨ Z_19), (Z_11 ∨· · ·∨ Z_51 ∨ Z_49 ∨· · ·∨ Z_19), (Z_15 ∨···∨ Z_55), (Z_44 ∨ Z_45 ∨ Z_46 ∨ Z_54 ∨ Z_55), when equal Z’s are identified via (i). Condition (iii), similarly, calls for 28 clauses such as (Z¯11∨Z¯12∨Z¯13), (Z¯11∨Z¯21∨Z¯31), (Z¯45∨Z¯55). Condition (vi) is enforced by 34,992 clauses epitomized by (S¯111∨Z¯11∨S¯122∨Z¯12∨S¯412∨Z¯41∨S¯421∨Z¯42).

For conditions (iv) and (v), we introduce auxiliary variables Vijk=Sijk ∧ Z¯ij, meaning that k is visible in (i, j); Rik = Vi_1k ∨ ··· ∨ Vi_9k, meaning that k is visible in row i; Cjk = V_1jk ∨··· ∨ V_9jk, meaning that k is visible in column j. Also Bbk = ∨〈i,j〉=b Vijk, meaning that k is visible in box b; here 〈i, j〉 = 1 + 3 ⌊(i − 1)/3⌋ + ⌊(j − 1)/3⌋. Then Pijk=Zij ∧ R¯ik ∧ C¯jk ∧ B¯〈i,j〉k means that k is a possible way to fill cell (i, j) without conflict. These 1701 auxiliary variables are defined with 8262 clauses.

Condition (iv) is enforced by nine 9-ary clauses for each i and j, stating that we mustn’t have exactly one of {Pij_1,..., Pij_9} true. Condition (v) is similar, enforced by three sets of 81 × 9 clauses of length 9; for example, one of those clauses is

(P417∨P427∨P437∨P517∨P¯527∨P537∨P617∨P627∨P637).

(“We aren’t obviously forced to put 7 into box 4 by using cell (5, 2).”)

Finally, some of the symmetry is usefully broken by asserting the unary clauses (S1kk)∧(Z¯11)∧(Z12). Altogether there are 2,471 variables, 58,212 clauses, 351,432 cells.

(This problem was suggested by Daniel Kroening. There are zillions of solutions, and about one in every five or six appears to be completable uniquely to the setting of the S variables. Thus we can obtain as many “hard sudoku” puzzles as we like, by adding additional unary clauses such as (S553)∧(Z¯17) more-or-less at random, then weeding out ambiguous cases via dancing links. The clauses are readily handled by Algorithms L or C, but they’re often too difficult for Algorithm D. That algorithm did, however, find the uniquely completable solution (a) below after only 9.3 Gμ of work.)

If we beef up condition (iii), insisting now that no box contains a row or column with more than one blank, condition (vi) becomes superfluous. We get solutions such as (b) below, remarkable for having no forced moves in spite of 58 visible clues, yet uniquely completable. That puzzle is, however, quite easy; only 2, 4, 7 are unplaced.

1....6.8.

1.3.56.89

1.3.5.7..

1.3.56.89

5.87214.6

59738.61.

.5.79...1

68.3.91.5

.6.38.2.1

68.1.93.5

7....125.

.9518.63.

84...3..5

956.318.7

..1..5.76

3.896..51

..5.6.8..

.315.896.

..5.7.1..

.195.836.

6..8...42

2.896.153

47.1..5..

56..319.8

3.6.48.2.

8.96.5.31

.185....7

.56.9381.

4.76321.8

.65.13298

5...87.1.

8.16.5.93

.8.5....4

31.89.5.6

..7.1.8.5

93.81.5.6

(a)

(b)

(c)

(d)

We might also try to strengthen conditions (iv) and (v) by requiring at least three ways to make each choice, not just two. Then we get solutions like (c) above. Unfortunately, however, that one is completable in 1237 ways! Even if we also strengthen condition (iii) as in (b), we get solutions like (d), which can be completed in 12 ways. No uniquely completable sudoku puzzles are known to have such ubiquitous threefold ambiguity.

516. This conjecture can be expressed in several equivalent forms. R. Impagliazzo and R. Paturi [J. Comp. Syst. Sci. 62 (2001), 367–375] defined sk = inf{ lg τ | we can know an algorithm to solve kSAT in τn steps}, and stated the exponential time hypothesis: s_3 > 0. They also defined s∞ = limk→∞ sk, and proved that sk ≤ (1 − d/k)s∞ for some positive constant d. They conjectured that s∞ = 1; this is the strong exponential time hypothesis. An alternative formulation [C. Calabro, R. Impagliazzo, and R. Paturi, IEEE Conf. on Comput. Complexity 21 (2006), 252–260] was found later: “If τ < 2, there is a constant α such that no knowable randomized algorithm can solve every SAT problem with ≤ αn clauses in fewer than τn steps, where n is the number of variables.”

517. (a) (Solution by Günter Rote.) Replace the jth ternary clause (lj∨lj′∨lj′′) by three ternary equations lj + aj + cj = 1, l¯j′+aj+bj=1, l¯j′′+cj+dj=1, where aj, bj, cj, and dj are new variables.

(b) Remove equations of length > 3 by using the fact that l_1 + ··· + lk = 1 if and only if l_1 + ··· + lj + t = 1 and lj+1+⋅⋅⋅+lk+t¯=1, where t is a new variable. Also, if a, b, c, and d are new variables with a+b+d=a+c+d¯=1, beef up short equations using l + l′ = 1 ⇔ l + l′ + a = 1 and l=1⇔l¯+b+c=1.

[Thomas J. Schaefer proved the NP-completeness of 1-in-3 SAT as a special case of considerably more general results, in STOC 10 (1978), 216–226.]

518. (a) A=(xyyyxyyyx), where x=−1010, y=11−11.

(b) Twice in the n variable rows and n variable columns; once in the 3m output rows and 3m input columns; never in the 3m input rows and 3m output columns.

(c) By (a), each way to choose 2s in different rows and columns contributes zero to the permanent unless, in every clause, the subset of chosen inputs is nonempty and matches the chosen outputs. In the latter case it contributes 16m2n. [See A. Ben-Dor and S. Halevi, Israel Symp. Theory of Computing Systems 2 (IEEE, 1993), 108–117.]

519. The unsatisfiable problem corresponding to D1 and D2 has median running time 2099 Mμ (losing to both factor_fifo and factor_lifo). The satisfiable one corresponding to D3 and D4 is unstable (as in Fig. 97), with median 903 Mμ (winning over both).

520. (Solution by Sven Mallach, 2015, using solvers X and Y, where X was CPLEX 12.6 and Y was GUROBI 6, both used with emphasis on mixed-integer-program feasibility, constant objective function, and solution limit 1.) With a time cutoff of 30 minutes on a single-threaded Xeon computer, neither X nor Y could solve any of the 46 problems A1, A2, C1, C2, C3, C4, C5, C6, C8, D1, D2, E1, E2, F1, F2, G1, G2, G5, G6, G7, G8, K7, K8, M5, M7, M8, O1, O2, P0, P1, P2, Q7, S3, S4, T5, T6, T7, T8, W2, W4, X1, X3, X5, X6, X7, X8. (In particular, this list includes P0, S4, and X1, which are extremely easy for Algorithm C.) On the other hand both X and Y solved the langford problems L3 and L4 — which were the toughest for Algorithm C — in less than a second.

Algorithm C performs about 20 Gμ per minute on a comparable Xeon. In these experiments it significantly outperformed the geometric methods except on problems K0, K1, K2, L3, L4, and P4 (and some easy cases such as B2).

Of course we must keep in mind that the particular clauses in Table 6 aren’t necessarily the best ways to solve the corresponding combinatorial problems with an IP solver, just as they aren’t necessarily the best encodings for a SAT solver. We are comparing here only black-box clause-solving speeds.

521. A variety of simple schemes have been surveyed by S. Jabbour, J. Lonlac, L. Saïs, and Y. Salhi, arXiv:1402.1956 [cs.AI] (2014), 13 pages.

522. For cycles of length T we can introduce 27T variables xyzt for 1 ≤ x, y, z ≤ 3 and 0 ≤ t < T , signifying that vertex (x, y, z) occupies slot t in the path. Binary exclusion clauses ¬xyzt∨¬x′y′zt′′, when xyz = x′y′z′ and t ≠ t′ or when xyz ≠ x′y′z′ and t = t′, ensure that no vertex appears twice in the path, and that no two vertices occupy the same slot. A valid path is specified via the adjacency clauses

¬xyzt∨∨{x′y′z(t+1) mod T′ | 1≤x′,y′,z′≤3 and |x′−x|+|y′−y|+|z′−z|=1}.

We represent the shadows by introducing 36 variables a!b∗, ba!∗, a!∗b, b∗a!, ∗a!b, ∗ba! for 1 ≤ a ≤ 2 and 1 ≤ b ≤ 3; here a!∗b (for example) means that the shadow of (x, z) coordinates has a transition between (a, b) and (a+1,b). These variables appear in ternary clauses such as (¬xyzt ∨¬(x+1)yzt ∨ x!∗z) ∧ (¬xyzt′ ∨ ¬(x+1)yzt′ ∨ x!y∗) whenever x < 3 and t′ ≡ t ± 1 (modulo T ). To exclude loops we append clauses like

¬1!1∗ ∨ ¬2!1∗ ∨ ¬31!∗ ∨ ¬32!∗ ∨ ¬2!3∗ ∨ ¬22!∗ ∨ ¬1!2∗ ∨ ¬11!∗;
this one excludes the loop in the example illustration. There are 39 such loop-defeating clauses, one for each of the 13 simple cycles in each shadow.

Finally we can break symmetry by asserting the unary clauses 121T−1, 111_0, 112_1 without loss of generality, after verifying that no solution can avoid all eight corners.

Clearly T must be an even number, because the graph is bipartite; also T < 27. If the method of exercise 12 is used for the exclusions, we obtain a total of 6264 clauses, 822 variables, and 17439 cells when T = 16; there are 9456 clauses, 1242 variables, and 26199 cells when T = 24. These clauses are too difficult for Algorithm D. But Algorithm L resolves them almost instantaneously for any given T ; they turn out to be satisfiable if and only if T = 24, and in that case there are two essentially different solutions. One of these cycles, due to John Rickard (who introduced this problem at Cambridge University, circa 1990), is beautifully symmetric, and it is illustrated on the cover of Peter Winkler’s book Mathematical Mind-Benders (2007). It can be represented by the delta sequence (3223¯1331¯2¯1123¯2¯2¯31¯3¯121¯1¯2¯), where ‘k’ and ‘k¯’ change coordinate k by +1 or −1. The other is unsymmetric and represented by (33212¯13¯1¯2212¯3231¯1¯3¯12¯1¯3¯2¯).

523. (Solution by Peter Winkler.) With coordinates (x, y, z) for 1 ≤ x ≤ m, 1 ≤ y ≤ n, 1 ≤ z ≤ 2, any cycle with loopless shadows must contain at least two steps (x, y, 1) −−− (x, y, 2) and (x′,y′, 1) −−− (x′,y′, 2). We can assume that x < x′ and that x′ − x is minimum. The m × 2 shadow contains (x, 1) −−− (x, 2) and (x′, 1) −−− (x′, 2), together with (say) the path (x, 1) −−−· · ·−−− (x′, 1), but without the edge (x″, 2) −−− (x″+1, 2) for some x″ with x ≤ x″ < x′. The unique shortest path from (x, y) to (x′,y′) in the m × n shadow contains some edge (x″,y″) −−− (x″+1,y″); hence (x″, y″, 1) −−− (x″+1, y″, 1) must occur twice in the cycle.

524. This problem involves clauses very much like those for a cyclic path, but simpler; we have T = 27 and no “wrap-around” conditions. With typically 1413 variables, 10410 clauses, and 28701 cells, Algorithm L shines again, needing only a gigamem or two to handle each of several cases that break symmetry based on starting and ending points. There are four essentially different solutions, each of which can be assumed to start at 111; one ends at 333, another at 133, another at 113, and the other at 223. Using the delta sequence notation above, they are: 3323¯23313¯2¯332¯3¯13323¯233 (which is reflected ternary code); 313¯1331¯1¯2113¯1¯31¯3¯2313¯1331¯1¯; 323¯2313¯2¯32¯3¯1323¯2332¯2¯1¯221¯2¯2¯; 11221¯1¯2¯131¯2112¯2¯1¯1¯311221¯1¯2¯1.

[Such paths, and more generally spanning trees that have loopless shadows, were invented in 1983 by Oskar van Deventer, who called them “hollow mazes”; see The Mathemagician and Pied Puzzler (1999), 213–218. His Mysterians puzzle is based on an amazing Hamiltonian path on P_5 □ P_5 □ P_5 that has loopless shadows.]

525. The author’s best solution, as of July 2015, had 100 variables, 400 clauses, and 1200 literals (cells); it was derived from Tseytin’s examples of exercise 245, applied to a more-or-less random 4-regular graph of girth 6 on 50 vertices. Tseytin’s construction, with one odd vertex and 49 even ones, yields 400 clauses of 4SAT, which are quite challenging indeed. It can be simplified to a 3SAT problem by insisting further that every even vertex must have degree exactly 2 in the subgraph specified by true edges. (See K. Markström, J. Satisfiability, Boolean Modeling and Comp. 2 (2006), 221–227).

That simplified problem still turned out to be fairly challenging: It was proved unsatisfiable by Algorithm L in 3.3 Tμ and by Algorithm C in 1.9 Tμ. (But by applying the endomorphisms of exercise 473, which broke symmetry by adding 142 clauses of length 6, the running time went down to just 263 Mμ and 949 Mμ, respectively.)

Another class of small-yet-difficult problems is worth mentioning, although it doesn’t fit the specifications of this exercise [see I. Spence, ACM J. Experimental Algorithmics 20 (2015), 1.4:1–1.4:14]: Every instance of 3D matching whose representation as an exact cover problem has 3n items and 5n options, with five options for each item and three items in each option, can be represented as a SAT problem in 3n variables, 10n binary clauses, and 2n quinary clauses, hence only 30n total literals. This 5SAT problem has the same number of literals as the 3SAT problem discussed above, when n = 40; yet it is considerably more difficult if the matching problem is unsatisfiable. (The problem of this kind that defeated all the SAT solvers in the 2014 competition corresponds to an instance of 3D matching that is solved almost instantaneously by the dancing links method: Algorithm 7.2.2.1X needs fewer than 60 Mμ to prove it unsatisfiable. On the other hand, if we encode that 3D matching problem with 3n quinary at-least-one and 3n · 10 binary at-most-one clauses, as in (13), instead of using only 2n for at-least-one and n · 10 for at-most-one, Algorithm L will be almost as good as dancing links.)

526. We prove by induction on |F | that it’s possible to leave at most w(F ) clauses unsatisfied, where w(F) = ∑C∉F 2−|C|: If all clauses of the multiset F are empty we have w(F) = |F|, and the result holds. Otherwise suppose the variable x appears in F . Let l = x if w({C | x∈C∈F})≥w({C | x¯∈C∈F}); otherwise l=x¯. A simple calculation shows that w(F | l) ≤ w(F ). [J. Computer and System Sciences 9 (1974), 256–278, Theorem 3.]

Appendix A—Tables of Numerical Quantities

Table 1
QUANTITIES THAT ARE FREQUENTLY USED IN STANDARD SUBROUTINES AND IN ANALYSIS OF COMPUTER PROGRAMS (40 DECIMAL PLACES)

2=1.41421 35623 73095 04880 16887 24209 69807 85697–

3=1.73205 08075 68877 29352 74463 41505 87236 69428+

5=2.23606 79774 99789 69640 91736 68731 27623 54406+

10=3.16227 76601 68379 33199 88935 44432 71853 37196–

23=1.25992 10498 94873 16476 72106 07278 22835 05703–

33=1.44224 95703 07408 38232 16383 10780 10958 83919–

24=1.18920 71150 02721 06671 74999 70560 47591 52930–

ln 2 = 0.69314 71805 59945 30941 72321 21458 17656 80755+

ln 3 = 1.09861 22886 68109 69139 52452 36922 52570 46475–

ln 10 = 2.30258 50929 94045 68401 79914 54684 36420 76011+

1/ln 2 = 1.44269 50408 88963 40735 99246 81001 89213 74266+

1/ln 10 = 0.43429 44819 03251 82765 11289 18916 60508 22944–

π = 3.14159 26535 89793 23846 26433 83279 50288 41972–

1° = π/180 = 0.01745 32925 19943 29576 92369 07684 88612 71344+

1/π = 0.31830 98861 83790 67153 77675 26745 02872 40689+

π^2 = 9.86960 44010 89358 61883 44909 99876 15113 53137–

π=Γ(1/2)=1.77245 38509 05516 02729 81674 83341 14518 27975+

Γ(1/3) = 2.67893 85347 07747 63365 56929 40974 67764 41287–

Γ(2/3) = 1.35411 79394 26400 41694 52880 28154 51378 55193+

e = 2.71828 18284 59045 23536 02874 71352 66249 77572+

1/e = 0.36787 94411 71442 32159 55237 70161 46086 74458+

e^2 = 7.38905 60989 30650 22723 04274 60575 00781 31803+

γ = 0.57721 56649 01532 86060 65120 90082 40243 10422–

ln π = 1.14472 98858 49400 17414 34273 51353 05871 16473–

ϕ = 1.61803 39887 49894 84820 45868 34365 63811 77203+

eγ = 1.78107 24179 90197 98523 65041 03107 17954 91696+

eπ/4 = 2.19328 00507 38015 45655 97696 59278 73822 34616+

sin 1 = 0.84147 09848 07896 50665 25023 21630 29899 96226–

cos 1 = 0.54030 23058 68139 71740 09366 07442 97660 37323+

–ζ′(2) = 0.93754 82543 15843 75370 25740 94567 86497 78979–

ζ(3) = 1.20205 69031 59594 28539 97381 61511 44999 07650–

ln ϕ = 0.48121 18250 59603 44749 77589 13424 36842 31352–

1/ln ϕ = 2.07808 69212 35027 53760 13226 06117 79576 77422–

–ln ln 2 = 0.36651 29205 81664 32701 24391 58232 66946 94543–

Table 2
QUANTITIES THAT ARE FREQUENTLY USED IN STANDARD SUBROUTINES AND IN ANALYSIS OF COMPUTER PROGRAMS (40 HEXADECIMAL PLACES)
The names at the left of the “=” signs are given in decimal notation.

0.1 = 0.1999 9999 9999 9999 9999 9999 9999 9999 9999 999A–

0.01 = 0.028F 5C28 F5C2 8F5C 28F5 C28F 5C28 F5C2 8F5C 28F6–

0.001 = 0.0041 8937 4BC6 A7EF 9DB2 2D0E 5604 1893 74BC 6A7F–

0.0001 = 0.0006 8DB8 BAC7 10CB 295E 9E1B 089A 0275 2546 0AA6+

0.00001 = 0.0000 A7C5 AC47 1B47 8423 0FCF 80DC 3372 1D53 CDDD+

0.000001 = 0.0000 10C6 F7A0 B5ED 8D36 B4C7 F349 3858 3621 FAFD–

0.0000001 = 0.0000 01AD 7F29 ABCA F485 787A 6520 EC08 D236 9919+

0.00000001 = 0.0000 002A F31D C461 1873 BF3F 7083 4ACD AE9F 0F4F+

0.000000001 = 0.0000 0004 4B82 FA09 B5A5 2CB9 8B40 5447 C4A9 8188–

0.0000000001 = 0.0000 0000 6DF3 7F67 5EF6 EADF 5AB9 A207 2D44 268E–

2=1.6A09 E667 F3BC C908 B2FB 1366 EA95 7D3E 3ADE C175+

3=1.BB67 AE85 84CA A73B 2574 2D70 78B8 3B89 25D8 34CC+

5=2.3C6E F372 FE94 F82B E739 80C0 B9DB 9068 2104 4ED8–

10=3.298B 075B 4B6A 5240 9457 9061 9B37 FD4A B4E0 ABB0–

23=1.428A 2F98 D728 AE22 3DDA B715 BE25 0D0C 288F 1029+

33=1.7137 4491 23EF 65CD DE74 16C5 6E32 67C0 A189 4C2B–

24=1.306F E0A3 1B71 52DE 8D5A 4630 5C85 EDEC BC27 3436+

ln 2 = 0.B172 17F7 D1CF 79AB C9E3 B398 03F2 F6AF 40F3 4326+

ln 3 = 1.193E A7AA D030 A976 A419 8D55 053B 7CB5 BE14 42DA–

ln 10 = 2.4D76 3776 AAA2 B05B A95B 58AE 0B4C 28A3 8A3F B3E7+

1/ln 2 = 1.7154 7652 B82F E177 7D0F FDA0 D23A 7D11 D6AE F552–

1/ln 10 = 0.6F2D EC54 9B94 38CA 9AAD D557 D699 EE19 1F71 A301+

π = 3.243F 6A88 85A3 08D3 1319 8A2E 0370 7344 A409 3822+

1° = π/180 = 0.0477 D1A8 94A7 4E45 7076 2FB3 74A4 2E26 C805 BD78–

1/π = 0.517C C1B7 2722 0A94 FE13 ABE8 FA9A 6EE0 6DB1 4ACD–

π^2 = 9.DE9E 64DF 22EF 2D25 6E26 CD98 08C1 AC70 8566 A3FE+

π=Γ(1/2)=1.C5BF 891B 4EF6 AA79 C3B0 520D 5DB9 383F E392 1547–

Γ(1/3) = 2.ADCE EA72 905E 2CEE C8D3 E92C D580 46D8 4B46 A6B3–

Γ(2/3) = 1.5AA7 7928 C367 8CAB 2F4F EB70 2B26 990A 54F7 EDBC+

e = 2.B7E1 5162 8AED 2A6A BF71 5880 9CF4 F3C7 62E7 160F+

1/e = 0.5E2D 58D8 B3BC DF1A BADE C782 9054 F90D DA98 05AB–

e^2 = 7.6399 2E35 376B 730C E8EE 881A DA2A EEA1 1EB9 EBD9+

γ = 0.93C4 67E3 7DB0 C7A4 D1BE 3F81 0152 CB56 A1CE CC3B–

ln π = 1.250D 048E 7A1B D0BD 5F95 6C6A 843F 4998 5E6D DBF4–

ϕ = 1.9E37 79B9 7F4A 7C15 F39C C060 5CED C834 1082 276C–

eγ = 1.C7F4 5CAB 1356 BF14 A7EF 5AEB 6B9F 6C45 60A9 1932+

eπ/4 = 2.317A CD28 E395 4F87 6B04 B8AB AAC8 C708 F1C0 3C4A+

sin 1 = 0.D76A A478 4867 7020 C6E9 E909 C50F 3C32 89E5 1113+

cos 1 = 0.8A51 407D A834 5C91 C246 6D97 6871 BD29 A237 3A89+

–ζ′(2) = 0.F003 2992 B55C 4F28 88E9 BA28 1E4C 405F 8CBE 9FEE+

ζ(3) = 1.33BA 004F 0062 1383 7171 5C59 E690 7F1B 180B 7DB1+

ln ϕ = 0.7B30 B2BB 1458 2652 F810 812A 5A31 C083 4C9E B233+

1/ln ϕ = 2.13FD 8124 F324 34A2 63C7 5F40 76C7 9883 5224 4685–

–ln ln 2 = 0.5DD3 CA6F 75AE 7A83 E037 67D6 6E33 2DBC 09DF AA82–

Several interesting constants with less common names have arisen in connection with the analyses in the present book. Those constants have been evaluated to 40 decimal places in Eq. 7.2.2.1–(86) and in the answer to exercise MPR–19(d).

Table 3
VALUES OF HARMONIC NUMBERS, BERNOULLI NUMBERS, AND FIBONACCI NUMBERS, FOR SMALL VALUES OF n

n

Hn

Bn

Fn

n

0

0

1

0

0

1

1

1/2

1

1

2

3/2

1/6

1

2

3

11/6

0

2

3

4

25/12

–1/30

3

4

5

137/60

0

5

5

6

49/20

1/42

8

6

7

363/140

0

13

7

8

761/280

–1/30

21

8

9

7129/2520

0

34

9

10

7381/2520

5/66

55

10

11

83711/27720

0

89

11

12

86021/27720

–691/2730

144

12

13

1145993/360360

0

233

13

14

1171733/360360

7/6

377

14

15

1195757/360360

0

610

15

16

2436559/720720

–3617/510

987

16

17

42142223/12252240

0

1597

17

18

14274301/4084080

43867/798

2584

18

19

275295799/77597520

0

4181

19

20

55835135/15519504

–174611/330

6765

20

21

18858053/5173168

0

10946

21

22

19093197/5173168

854513/138

17711

22

23

444316699/118982864

0

28657

23

24

1347822955/356948592

–236364091/2730

46368

24

25

34052522467/8923714800

0

75025

25

26

34395742267/8923714800

8553103/6

121393

26

27

312536252003/80313433200

0

196418

27

28

315404588903/80313433200

–23749461029/870

317811

28

29

9227046511387/2329089562800

0

514229

29

30

9304682830147/2329089562800

8615841276005/14322

832040

30

For any x, let Hx=Σn≥1(1n–1n+x). Then

H1/2=2–2 ln 2,H1/3=3–12π/3–32 ln 3,H2/3=32+12π/3–32 ln 3,H1/4=4–12π–3 ln 2,H3/4=43+12π–3 ln 2,H1/5=5–12πϕ3/25–1/4–54 ln 5–125 ln ϕ,H2/5=52–12πϕ3/25–1/4–54 ln 5+125 ln ϕ,H3/5=53+12πϕ–3/25–1/4–54 ln 5+125 ln ϕ,H4/5=54+12πϕ3/25–1/4–54 ln 5–125 ln ϕ,H1/6=6–12π3–2 ln 2–32 ln 3,H5/6=65+12π3–2 ln 2–32 ln 3,
and, in general, when 0 < p < q (see exercise 1.2.9–19),

Hp/q=qp–π2cotpqπ–ln2q+2Σ1≤n<q/2cos2pnqπ⋅lnsinnqπ.

Appendix B—Index to Notations

In the following formulas, letters that are not further qualified have the following significance:

j, k

integer-valued arithmetic expression

m, n

nonnegative integer-valued arithmetic expression

p, q

binary-valued arithmetic expression (0 or 1)

x, y

real-valued arithmetic expression

z

complex-valued arithmetic expression

f

integer-valued, real-valued, or complex-valued function

G, H

graph

S, T

set or multiset

F, G

family of sets

u, v

vertex of a graph

α, β

string of symbols

The place of definition is either a page number in the present volume or a section number in another volume. Many other notations, such as Kn for the complete graph on n vertices, appear in the main index at the close of this book. See also ‘Notational conventions’ in that index.

Formal symbolism

Meaning

Where defined

V← E

give variable V the value of expression E

§1.1

U ↔ V

interchange the values of variables U and V

§1.1

An or A[n]

the nth element of linear array A

§1.1

Amn or A[m, n]

the element in row m and column n of rectangular array A

§1.1

(R? a: b)

conditional expression: denotes a if relation R is true, b if R is false

336

[R]

characteristic function of relation R: (R? 1: 0)

§1.2.3

δjk

Kronecker delta: [j = k]

§1.2.3

[zn] f(z)

coefficient of zn in power series f(z)

§1.2.9

z_1 + z_2 + ... + zn

sum of n numbers (even when n is 0 or 1)

§1.2.3

a_1a_2 ... an

product or string or vector of n elements

(x_1,...,xn)

vector of n elements

〈x_1x_2 ... x_2k–1〉

median value (the middle value after sorting)

§7.1.1

∑R(k) f(k)

sum of all f(k) such that relation R(k) is true

§1.2.3

ΠR(k) f(k)

product of all f(k) such that relation R(k) is true

§1.2.3

minR(k) f(k)

minimum of all f(k) such that relation R(k) is true

§1.2.3

maxR(k) f(k)

maximum of all f(k) such that relation R(k) is true

§1.2.3

∪R(k) S(k)

union of all S(k) such that relation R(k) is true

Σk=abf(k)

shorthand for ∑a≤k≤b f(k)

§1.2.3

{a | R(a)}

set of all a such that relation R(a) is true

∑{f(k) | R(k)}

another way to write ∑R(k) f(k)

{a_1,a_2,...,an}

the set or multiset {ak | 1 ≤ k ≤ n}

[x..y]

closed interval: {a | x ≤ a ≤ y}

§1.2.2

(x..y)

open interval: {a | x < a < y}

§1.2.2

[x..y)

half-open interval: {a | x ≤ a < y}

§1.2.2

(x..y]

half-closed interval: {a | x < a ≤ y}

§1.2.2

|S|

cardinality: the number of elements in S

|x|

absolute value of x: (x ≥ 0? x: –x)

|z|

absolute value of z:zz¯

§1.2.2

|α|

length of α: m if α = a_1a_2 ... am

|l|

base variable of literal l:|υ|=|υ¯|=υ

186

⌊x⌋

floor of x, greatest integer function: maxk≤x k

§1.2.4

⌈x⌉

ceiling of x, least integer function: mink≥x k

§1.2.4

x mod y

mod function: (y = 0? x: x – y ⌊x/y⌋)

§1.2.4

{x}

fractional part (used in contexts where a real value, not a set, is implied): x mod 1

§1.2.11.2

x ≡ x′ (modulo y)

relation of congruence: x mod y = x′ mod y

§1.2.4

j\k

j divides k: k mod j = 0 and j > 0

§1.2.4

S \ T

set difference: {s | s in S and s not in T}

S \ t

shorthand for S \ {t}

G \ U

G with vertices of the set U removed

§7

G \ v

G with vertex v removed

§7

G \ e

G with edge e removed

§7

G/e

G with edge e shrunk to a point

§7.2.1.6

S ∪ t

shorthand for S ∪ {t}

multiset sum; e.g.,

§4.6.3

gcd(j, k)

greatest common divisor: (j=k=0? 0: maxd\j,d\k d)

§1.1

j ⊥ k

j is relatively prime to k: gcd(j, k) = 1

§1.2.4

AT

transpose of rectangular array A: AT [j, k] = A[k, j]

αR

left-right reversal of string α

αT

conjugate of partition α

§7.2.1.4

xy

x to the y power (when x > 0): ey ln x

§1.2.2

xk

x to the k power: (k≥0?Πj=0k–1x:1/x–k)

§1.2.2

x–

inverse (or reciprocal) of x: x–1

§1.3.3

xk¯

x to the k rising: Γ(x+k)/Γ(k)=(k≥0?Πj=0k–1(x+j):1/(x+k)–k¯)

§1.2.5

xk

x to the k falling: x!/(x–k)!=(k≥0?Πj=0k–1(x–j):1/(x–k)––k)

§1.2.5

n!

n factorial: Γ(n + 1) = nn

§1.2.5

(xk)

binomial coefficient: (k < 0? 0:xk/k!)

§1.2.6

(nn1,...,nm)

multinomial coefficient (when n = n_1 + ... + nm)

§1.2.6

[nm]

Stirling cycle number: ∑0<k_1<...<kn–m<n k_1 ...kn–m

§1.2.6

{nm}

Stirling subset number: ∑1≤k_1≤...≤kn–m≤m k_1 ... kn–m

§1.2.6

〈nm〉

Eulerian number: Σk=0m(–1)k(n+1k)(m+1–k)n

§5.1.3

|nm|

m-part partitions of n: ∑1≤k_1≤...≤km [k_1 + ... + km = n]

§7.2.1.4

(...a_1a_0.a–1 ...)b

radix-b positional notation: ∑k akbk

§4.1

Rz

real part of z

§1.2.2

ℑz

imaginary part of z

§1.2.2

z¯

complex conjugate: Rz − i ℑz

§1.2.2

¬p or ~p or p¯

complement: 1 – p

§7.1.1

~x or x¯

bitwise complement

§7.1.3

p ∧ q

Boolean conjunction (and): pq

§7.1.1

x ∧ y

minimum: min{x, y}

§7.1.1

x & y

bitwise AND

§7.1.3

p ∨ q

Boolean disjunction (or): p¯q¯¯

§7.1.1

x ∨ y

maximum: max{x, y}

§7.1.1

x | y

bitwise OR

§7.1.3

p ⊕ q

Boolean exclusive disjunction (xor): (p + q) mod 2

§7.1.1

x ⊕ y

bitwise XOR

§7.1.3

x–˙y

saturated subtraction, x monus y: max{0, x – y}

§1.3.1′

x ≪ k

bitwise left shift: ⌊2kx⌋

§7.1.3

x ≫ k

bitwise right shift: x ≪ (–k)

§7.1.3

x ‡ y

“zipper function” for interleaving bits, x zip y

§7.1.3

logb x

logarithm, base b, of x (defined when x > 0, b > 0, and b ≠ 1): the y such that x = by

§1.2.2

ln x

natural logarithm: loge x

§1.2.2

lg x

binary logarithm: log_2 x

§1.2.2

λn

binary logsize (when n > 0): ⌊lg n⌋

§7.1.3

exp x

exponential of x:ex=Σk=0∞xk/k!

§1.2.9

ρn

ruler function (when n > 0): max2m\n m

§7.1.3

νn

sideways sum (when n ≥ 0): ∑k≥0 ((n ≫ k)&1)

§7.1.3

〈Xn〉

the infinite sequence X_0, X_1, X_2, ... (here the letter n is part of the symbolism)

§1.2.9

f′(x)

derivative of f at x

§1.2.9

f″(x)

second derivative of f at x

§1.2.10

Hn(x)

harmonic number of order x:Σk=1n1/kx

§1.2.7

Hn

harmonic number: Hn(1)

§1.2.7

Fn

Fibonacci number: (n ≤ 1? n: Fn–1 + Fn–2)

§1.2.8

Bn

Bernoulli number: n![zn] z/(1 – e–z)

§1.2.11.2

det(A)

determinant of square matrix A

§1.2.3

sign(x)

sign of x: [x > 0] – [x < 0]

ζ(x)

zeta function: limn→∞Hn(x) (when x > 1)

§1.2.7

Γ(x)

gamma function: (x – 1)! = γ(x, ∞)

§1.2.5

γ(x, y)

incomplete gamma function: ∫0ye–ttx–1dt

§1.2.11.3

γ

Euler’s constant: –Γ′(1) = limn→∞(Hn – ln n)

§1.2.7

e

base of natural logarithms: ∑n≥0 1/n!

§1.2.2

π

circle ratio: 4 ∑n≥0(–1)n/(2n + 1)

§1.2.2

∞

infinity: larger than any number

Λ

null link (pointer to no address)

§2.1

θ

empty set (set with no elements)

∊

empty string (string of length zero)

∊

unit family: {θ}

§7.1.4

ϕ

golden ratio: (1+5)/2

§1.2.8

φ(n)

Euler’s totient function: Σk=0n–1[k⊥n]

§1.2.4

x ≈ y

x is approximately equal to y

§1.2.5

G ≅ H

G is isomorphic to H

§7

O(f(n))

big-oh of f(n), as the variable n → ∞

§1.2.11.1

O(f(z))

big-oh of f(z), as the variable z → 0

§1.2.11.1

Ω(f(n))

big-omega of f(n), as the variable n → ∞

§1.2.11.1

Θ(f(n))

big-theta of f(n), as the variable n → ∞

§1.2.11.1

G¯

complement of graph (or uniform hypergraph) G

§7

GT

converse of digraph G (change ‘––→’ to ‘←––’)

§7.2.2.3

G | U

G restricted to the vertices of set U

§7

u––– v

u is adjacent to v

§7

u –/– v

u is not adjacent to v

§7

u–→ v

there is an arc from u to v

§7

u–→* v

transitive closure: v is reachable from u

§7.1.3

d(u, v)

distance from u to v

§7

G ∪ H

union of G and H

§7

G ⊕ H

direct sum (juxtaposition) of G and H

§7

G–– H

join of G and H

§7

G––→ H

directed join of G and H

§7

G □ H

Cartesian product of G and H

§7

G ⊗ H

direct product (conjunction) of G and H

§7

G ⊠ H

strong product of G and H

§7

G Δ H

odd product of G and H

§7

G ο H

lexicographic product (composition) of G and H

§7

ej

elementary family: {{j}}

§7.1.4

℘

universal family: all subsets of a given universe

§7.1.4

F ∪ G

union of families: {S | S ∈ F or S ∈ G}

§7.1.4

F ∩ G

intersection of families: {S | S ∈ F and S ∈ G}

§7.1.4

F \ G

difference of families: {S | S ∈ F and S ∉ G}

§7.1.4

F ⊕ G

symmetric difference of families: (F \ G) ∪ (G \ F)

§7.1.4

F ⊔ G

join of families: {S ∪ T | S ∈ F, T ∈ G}

§7.1.4

F ⊓ G

meet of families: {S ∩ T | S ∈ F, T ∈ G}

§7.1.4

F ⊞ G

delta of families: {S ⊕ T | S ∈ F, T ∈ G}

§7.1.4

F/G

quotient (cofactor) of families

§7.1.4

F mod G

remainder of families: F \ (G ⊔ (F/G))

§7.1.4

F § k

symmetrized family, if F = ej_1 ∪ ej_2 ∪ ... ∪ ejn

§7.1.4

F↑

maximal elements of F: {S ∈ F | T ∈ F and S ⊆ T implies S = T}

§7.1.4

F↓

minimal elements of F: {S ∈ F | T ∈ F and S ⊇ T implies S = T}

§7.1.4

F ↗ G

nonsubsets: {S ∈ F | T ∈ G implies S ⊈ T}

§7.1.4

F ↘ G

nonsupersets: {S ∈ F | T ∈ G implies S ⊉ T}

§7.1.4

F ↙ G

subsets: {S ∈ F | T ∈ G implies S ⊆ T} = F \ (F ↗ G)

§7.1.4

F ↖ G

supersets: {S ∈ F | T ∈ G implies S ⊇ T} = F \ (F ↘ G)

§7.1.4

X · Y

dot product of vectors: x_1y_1 + x_2y_2 + ... + xnyn, if X = x_1x_2 ... xn and Y = y_1y_2 ... yn

§7

X ⊆ Y

containment of vectors: xk ≤ yk for 1 ≤ k ≤ n, if X = x_1x_2 ... xn and Y = y_1y_2 ...yn

§7.1.3

α(G)

independence number of G

§7

γ(G)

domination number of G

461

κ(G)

vertex connectivity of G

§7.4.1.3

λ(G)

edge connectivity of G

§7.4.1.3

ν(G)

matching number of G

§7.5.5

χ(G)

chromatic number of G

§7

ω(G)

clique number of G

§7

c(G)

number of spanning trees of G

§7.2.1.6

C′ ⋄ C″

resolvent of clauses C′ and C″

336

Pr(S(X))

probability that statement S(X) is true, when X is a random variable

§1.2.10

E X

expected value of the random variable X: ∑x x Pr(X = x)

§1.2.10

var X

variance of the random variable X: E((X – E X)2)

2

Pr(A|B)

conditional probability of A given B: Pr(A and B)/Pr(B)

1

E(X |Y)

expected value of X given Y

3

▮

end of algorithm, program, or proof

§1.1

In the end, however, I did put in one equation, Einstein’s famous equation, E = mc^2.
I hope that this will not scare off half of my potential readers.

— STEPHEN HAWKING, A Brief History of Time (1987)

Appendix C—Index to Algorithms and Theorems

Algorithm 7.2.2B, 30.

Algorithm 7.2.2B*, 32.

Algorithm 7.2.2C, 45–46.

Algorithm 7.2.2E, 49.

Corollary 7.2.2E, 48.

Theorem 7.2.2E, 48.

Algorithm 7.2.2L, 35.

Algorithm 7.2.2O, 403.

Algorithm 7.2.2R, 414.

Algorithm 7.2.2R′, 414.

Algorithm 7.2.2W, 33.

Algorithm 7.2.2.1C, viii, 90.

Algorithm 7.2.2.1C$, viii, 113, 116–118.

Lemma 7.2.2.1D, 105.

Theorem 7.2.2.1E, 100.

Algorithm 7.2.2.1G, 423.

Algorithm 7.2.2.1I, 417.

Algorithm 7.2.2.1M, viii, 97–98.

Algorithm 7.2.2.1N, 128.

Algorithm 7.2.2.1P, viii, 110.

Theorem 7.2.2.1S, 108.

Algorithm 7.2.2.1X, viii, 69.

Algorithm 7.2.2.1X§, 112, 116–118.

Algorithm 7.2.2.1Z, viii, 120.

Algorithm 7.2.2.2A, viii, 212–213, 572.

Algorithm 7.2.2.2A*, 572–573.

Algorithm 7.2.2.2B, viii, 215.

Lemma 7.2.2.2B, 242.

Theorem 7.2.2.2B, 243.

Algorithm 7.2.2.2C, viii, 252.

Theorem 7.2.2.2C, 236–238.

Algorithm 7.2.2.2D, viii, 217–218.

Algorithm 7.2.2.2E, 360.

Corollary 7.2.2.2E, 297.

Theorem 7.2.2.2E, 295.

Algorithm 7.2.2.2F, 569.

Theorem 7.2.2.2F, 270.

Theorem 7.2.2.2G, 254.

Algorithm 7.2.2.2I, 245.

Theorem 7.2.2.2J, 266.

Algorithm 7.2.2.2K, 580–581.

Theorem 7.2.2.2K, 274.

Algorithm 7.2.2.2L, viii, 222–223.

Algorithm 7.2.2.2L^0, 223.

Algorithm 7.2.2.2L′, viii, 576.

Lemma 7.2.2.2L, 266.

Theorem 7.2.2.2L, 266.

Algorithm 7.2.2.2M, 266–267.

Theorem 7.2.2.2M, 267.

Algorithm 7.2.2.2P, 261.

Algorithm 7.2.2.2P′, 605.

Program 7.2.2.2P′, 606.

Algorithm 7.2.2.2R, 330.

Theorem 7.2.2.2R, 240.

Algorithm 7.2.2.2S, viii, 277.

Theorem 7.2.2.2S, 271.

Algorithm 7.2.2.2T, 613–614.

Theorem 7.2.2.2U, 262–263.

Algorithm 7.2.2.2W, viii, 263–264, 606.

Corollary 7.2.2.2W, 263.

Algorithm 7.2.2.2X, 228–229.

Algorithm 7.2.2.2Y, 230.

There is a curious poetical index to the Iliad in Pope’s Homer, referring to all the places in which similes are used.

— HENRY B. WHEATLEY, What is an Index? (1878)

Appendix D—Index to Combinatorial Problems

The purpose of this appendix is to present concise descriptions of the major problems treated in the present book, and to associate each problem description with the name under which it can be found in the main index. Some of these problems can be solved efficiently, while others appear to be very difficult in general although special cases might be easy. No indication of problem complexity is given here.

Combinatorial problems have a chameleon-like tendency to assume many forms. For example, certain properties of graphs and hypergraphs are equivalent to other properties of 0–1 matrices; and an m × n matrix of 0s and 1s can itself be regarded as a Boolean function of its index variables (i, j), with 0 representing FALSE and 1 representing TRUE. Each problem also has many flavors: We sometimes ask only whether a solution to certain constraints exists at all; but usually we ask to see at least one explicit solution, or we try to count the number of solutions, or to visit them all. Often we require a solution that is optimum in some sense.

In the following list — which is intended to be helpful but by no means complete — each problem is presented in more-or-less formal terms as the task of “finding” some desired objective. This characterization is then followed by an informal paraphrase (in parentheses and quotation marks), and perhaps also by further comments.

Any problem that is stated in terms of directed graphs is automatically applicable also to undirected graphs, unless the digraph must be acyclic, because an undirected edge u ––– v is equivalent to the two directed arcs u → v and v → u.

• Satisfiability: Given a Boolean function f of n Boolean variables, find Boolean values x_1, ... , xn such that f(x_1,...,xn) = 1. (“If possible, show that f can be true.”)

• kSAT: The satisfiability problem when f is the conjunction of clauses, where each clause is a disjunction of at most k literals xj or x¯j. (“Can all the clauses be true?”) The cases 2SAT and 3SAT are most important. Another significant special case arises when f is a conjunction of Horn clauses, each having at most one nonnegated literal xj.

• Boolean chain: Given one or more Boolean functions of n Boolean values x_1, ... , xn, find xn+1, ... , xN such that each xk for n < k ≤ N is a Boolean function of xi and xj for some i < k and j < k, and such that each of the given functions is either constant or equal to xl for some l ≤ N. (“Construct a straight-line program to evaluate a given set of functions, sharing intermediate values.”) (“Build a circuit to compute a given collection of outputs from the inputs 0, 1, x_1, ... , xn, using 2-input Boolean gates with unlimited fanout.”) The goal is usually to minimize N.

• Broadword chain: Like a Boolean chain, but using bitwise and/or arithmetic operations on integers modulo 2d instead of Boolean operations on Boolean values; the given value of d can be arbitrarily large. (“Work on several related problems at once.”)

• Boolean programming: Given a Boolean function f of n Boolean variables, together with given weights w_1, ... , wn, find Boolean values x_1, ... , xn such that f(x_1,...,xn) = 1 and w_1x_1 + ... + wnxn is as large as possible. (“How can f be satisfied with maximum payoff?”)

• Matching: Given a graph G, find a set of disjoint edges. (“Pair up the vertices so that each vertex has at most one partner.”) The goal is usually to find as many edges as possible; a “perfect matching” includes all the vertices. In a bipartite graph with m vertices in one part and n vertices in the other, matching is equivalent to selecting a set of 1s in an m × n matrix of 0s and 1s, with at most one selected in each row and at most one selected in each column.

• Assignment problem: A generalization of bipartite matching, with weights associated with each edge; the total weight of the matching should be maximized. (“What assignment of people to jobs is best?”) Equivalently, we wish to select elements of an m × n matrix, at most one per row and at most one per column, so that the sum of selected elements is as large as possible.

• Covering: Given a matrix Ajk of 0s and 1s, find a set of rows R such that we have ∑j∈R Ajk > 0 for all k. (“Mark a 1 in each column and select all rows that have been marked.”) Equivalently, find an implicant of a monotone Boolean function, given its clauses. The goal is usually to minimize |R|.

• Exact cover: Given a matrix Ajk of 0s and 1s, find a set of rows R such that ∑j∈R Ajk = 1 for all k. (“Cover with mutually orthogonal rows.”) The perfect matching problem is equivalent to finding an exact cover of the transposed incidence matrix.

• Independent set: Given a graph or hypergraph G, find a set of vertices U such that the induced graph G | U has no edges. (“Choose unrelated vertices.”) The goal is usually to maximize |U|. Classical special cases include the 8 queens problem, when G is the graph of queen moves on a chessboard, and the no-three-on-a-line problem.

• Clique: Given a graph G, find a set of vertices U such that the induced graph G | U is complete. (“Choose mutually adjacent vertices.”) Equivalently, find an independent set in ~G. The goal is usually to maximize |U|.

• Vertex cover: Given a graph or hypergraph, find a set of vertices U such that every edge includes at least one vertex of U. (“Mark some vertices so that no edge remains unmarked.”) Equivalently, find a covering of the transposed incidence matrix. Equivalently, find U such that V \ U is independent, where V is the set of all vertices. The goal is usually to minimize |U|.

• Dominatin g set: Given a graph, find a set of vertices U such that every vertex not in U is adjacent to some vertex of U. (“What vertices are within one step of them all?”) The classic 5-queens problem is the special case when G is the graph of queen moves on a chessboard.

• Kernel: Given a directed graph, find an independent set of vertices U such that every vertex not in U is the predecessor of some vertex of U. (“In what independent positions of a 2-player game can your opponent force you to remain?”) If the graph is undirected, a kernel is equivalent to a maximal independent set, and to a dominating set that is both minimal and independent.

• Coloring: Given a graph, find a way to partition its vertices into k independent sets. (“Color the vertices with k colors, never giving the same color to adjacent points.”) The goal is usually to minimize k.

• Shortest path: Given vertices u and v of a directed graph in which weights are associated with every arc, find the smallest total weight of an oriented path from u to v. (“Determine the best route.”)

• Longest path: Given vertices u and v of a directed graph in which weights are associated with every arc, find the largest total weight of a simple oriented path from u to v. (“What route meanders the most?”)

• Reachability: Given a set of vertices U in a directed graph G, find all vertices v such that u →* v for some u ∈ U. (“What vertices occur on paths that start in U?”)

• Spanning tree: Given a graph G, find a free tree F on the same vertices, such that every edge of F is an edge of G. (“Choose just enough edges to connect up all the vertices.”) If weights are associated with each edge, a minimum spanning tree is a spanning tree of smallest total weight.

• Hamiltonian path: Given a graph G, find a path P on the same vertices, such that every edge of P is an edge of G. (“Discover a path that encounters every vertex exactly once.”) This is the classic knight’s tour problem when G is the graph of knight moves on a chessboard. When the vertices of G are combinatorial objects — for example, tuples, permutations, combinations, partitions, or trees — that are adjacent when they are “close” to each other, a Hamiltonian path is often called a Gray code.

• Hamiltonian cycle: Given a graph G, find a cycle C on the same vertices, such that every edge of C is an edge of G. (“Discover a path that encounters every vertex exactly once and returns to the starting point.”)

• Traveling Salesrep Problem: Find a Hamiltonian cycle of smallest total weight, when weights are associated with each edge of the given graph. (“What’s the cheapest way to visit everything?”) If the graph has no Hamiltonian cycle, we extend it to a complete graph by assigning a very large weight W to every nonexistent edge.

• Topological sorting: Given a directed graph, find a way to label each vertex x with a distinct number l(x) in such a way that x → y implies l(x) < l(y). (“Place the vertices in a row, with each vertex to the left of all its successors.”) Such a labeling is possible if and only if the given digraph is acyclic.

• Optimum linear arrangement: Given a graph, find a way to label each vertex x with a distinct integer l(x), such that ∑u—v |l(u) – l(v)| is as small as possible. (“Place the vertices in a row, minimizing the sum of the resulting edge lengths.”)

• Kna psack problem: Given a sequence of weights w_1, ... , wn, a threshold W, and a sequence of values v_1, ... , vn, find K ⊆ {1,...,n} such that ∑k∈K wk ≤ W and ∑k∈K vk is maximum. (“How much value can be carried?”)

• Orthogonal array: Given positive integers m and n, find an m × n^2 array with entries Ajk ∈ {0, 1,...,n – 1} and with the property that j ≠ j′ and k ≠ k′ implies (Ajk,Aj′k) ≠ (Ajk′, Aj′k′). (“Construct m different n × n matrices of n-ary digits in such a way that all n^2 possible digit pairs occur when any two of the matrices are superimposed.”) The case m = 3 corresponds to a latin square, and the case m > 3 corresponds to m – 2 mutually orthogonal latin squares.

• Nearest common ancestor: Given nodes u and v of a forest, find w such that every inclusive ancestor of u and of v is also an inclusive ancestor of w. (“Where does the shortest path from u to v change direction?”)

• Range minimum query: Given a sequence of numbers a_1, ... , an, find the minimum elements of each subinterval ai,...,aj for 1 ≤ i < j ≤ n. (“Solve all possible queries concerning the minimum value in any given range.”) Exercises 150 and 151 of Section 7.1.3 show that this problem is equivalent to finding nearest common ancestors.

• Universal cycle: Given b, k, and N, find a cyclic sequence of elements x_0, x_1, ..., xN–1, x_0, ... of b-ary digits {0, 1,..., b–1} with the property that all combinatorial arrangements of a particular kind are given by the consecutive k-tuples x_0x_1 ...xk–1, x_1x_2 ...xk, ... , xN–1x_0 ...xk–2. (“Exhibit all possibilities in a circular fashion.”) The result is called a de Bruijn cycle if N = bk and all possible k-tuples appear; it’s a universal cycle of combinations if N=(bk) and if all k-combinations of b things appear; and it’s a universal cycle of permutations if N = b!, k = b–1, and if all (b–1)-variations appear as k-tuples.

In most cases we have been able to give a set-theoretic definition that describes the problem completely, although the need for conciseness has often led to some obscuring of the intuition behind the problem.

— M. R. GAREY and D. S. JOHNSON, A List of NP-Complete Problems (1979)

Appendix E—Answers to Puzzles in the Answers

All answers here refer to exercises in Section 7.2.2.1.

(see answer 52)

(see answer 58)

Click here to view code image

SEVENTH, FOURTEEN, FIGHTER, REINVENT, VENTURES;
NONE, FORGIVEN, FORGIVES, UNTHRONE;
UNDOERS, FOUNDERS, CONDORS, TRIODES, ROUNDEST,
SECONDO, CERTIFY, FORTIFY, EXTRUDES.

(see answer 112)

(see answer 173)

(see answer 174)

(see answer 282)

(see answer 302(c))

red bot = 188185335,mid = 179279435,top = 279266446; green bot = 599869866,mid = 557877443,top = 112122433

(see answer 337)

(see answer 395)

(see answer 396)

(see answer 403)

(see answer 407)

(see answer 408)

(see answer 409)

(see answer 411)

(see answer 415)

(see answer 416)

(see answer 418)

(see answer 424)

(see answer 426)

(see answer 427)

(see answer 431)

(see answer 430(a))

(see answer 435)

(see answer 447)

(see exercise 449)

(see answer 448)

By my troth, we that have good wits, have much to answer for.

— SHAKESPEARE (As You Like It, Act V, Scene 1, Line 11)

Index and Glossary

There is an easy index,
so you can find whatever you wish without delay.

— McCall’s Cook Book (1963)

When an index entry refers to a page containing a relevant exercise, see also the answer to that exercise for further information. An answer page is not indexed here unless it refers to a topic not included in the statement of the exercise.

# (number sign or hash mark, can indicate hexadecimal constants like # c0ffee), x.

∂S (boundary set), 242–243, 338, 364.

⇒: Implies.

⇔: If and only if.

∅ (the empty set), 549, 594.

0-origin indexing, 74, 424.

0–1 matrices, see Matrices of 0s and 1s.

{0, 1, 2} matrices, 146.

{0, 1, 2, 3} matrices, 467.

1×1×1 cube, 82.

1-in-3 SAT problem, 367.

1SAT problem, 233, 332.

2×2×2 cube, 139, 457.

2-colorability of hypergraphs, 549.

2-letter block codes, 57.

2-letter postal codes, 78, 114.

2-letter words of English, 36, 56, 413.

2-regular graphs, 53, 59, 146.

2D matching problem (2DM), 103, see Bipartite matching problems.

2SAT problem, 233, 235–238, 261–262, 264, 285, 328, 331, 333, 341–343, 351, 545, 630, 667.

3×3×3 cube, 84–85, 166, 368, 458, 515; see also Soma cube.

3-letter words of English, 36, 56, 413.

3-regular graphs, 331, 338, 595.

3CNF, 187, 332.

3D matching problem (3DM), 103, 131, 149, 318, 589, 654–655.

3D visualizations, 300–302.

3SAT problem, 187–188, 231–235, 243, 244, 262–264, 277–278, 315, 319, 330, 332–335, 337, 366–368, 438, 595, 667.

4×4×4 cube, 508.

4-cycles, 107, 293–294, 362, 473, 589, 638, 641.

4-fold symmetry, 171, 174, 322, 402, 465, 528, 546.

4-letter codewords, 37–46, 57.

4-letter words of English, 36, 56, 94–95, 152, 413, 437.

4-regular graphs, 654.

4D matching problem (4DM), 103.

4SAT problem, 233, 235, 334, 654.

5×5×5 cube, 167.

5-letter words of English, 36–37, 56, 59, 62, 94–95, 133–134, 136, 145, 152, 156, 183, 402.

5-queens problem, 93–94, 144, 668.

5SAT problem, 235, 242, 588, 655.

6-color cubes, 142–143, 457.

6-letter and k-letter words of English, 36–37, 56, 402, 413.

6SAT problem, 235.

7×7×7 cube, 173, 527.

7SAT problem, 235, 335.

8-fold symmetry, 174, 180, 322, 429, 489, 535, 546, 562.

8-neighbors (king moves), 145–146, 176, 503.

8 queens problem, 31–32, 47–48, 53–54, 209, 421, 646, 668; see n queens problem.

9 queens problem, 129.

12-tone rows, 135.

16 queens problem, 48, 72, 73, 112, 152, 155.

60°-rotational symmetry, 91, 528.

64 queens problem, 48, 477.

90°-rotational symmetry, 55, 126, 171, 174, 322, 428–429, 452, 478, 497, 505, 546, 566, 639.

100 test cases, vii, viii, 297–308, 311, 366, 368.

120°-rotational symmetry, 528.

180°-rotational symmetry (central symmetry), 126, 143, 171, 174, 428–429, 450, 454, 464–465, 490, 495, 528.

666 (number of the beast), 61, 159.

∞ queens problem, 127.

γ (Euler’s constant), 373, 656–657.

as source of “random” data, 47.

δf, Δf (parameters for flushing), 260–261, 342.

δp, Δp (parameters for purging), 258–259, 309–311, 650.

∈ (the empty clause), 187, 211, 549, 655.

∈ (the empty string), 268–269, 663.

∈ (the tolerance for convergence), 277–278.

ε (offset in heuristic scores), 310, 577.

λx (⌊lg x⌋), x, 532.

Λ (the null link), 407–408, 414.

μ(C) (clause complexity), 243, 337.

νx (1s count), see Sideways sum.

π (circle ratio), see Pi.

ϖn (the nth Bell number), 15, 101–102, 147–148, 467, 481, 530.

ρ (damping factor for reinforcement), 277–278.

ρ (damping factor for variable activity), 251, 309–311, 339, 650.

ρx (ruler function), x, 126, 344, 532, 609.

ϱ (damping factor for clause activity), 258, 309–311, 650.

τ parameter, 309–311, 599, 650.

τ (a, b) function, 331.

ϕ (golden ratio), 12, 121, 127, 330, 331, 344, 370, 451, 615, 656–657.

as source of “random” data, 47.

χ-critical graph, 137.

ψ (agility threshold), 260–261, 308–311, 604, 650.

ψ (confidence level), 277, 619.

A priori versus a posteriori probabilities, 25, 407.

a.s.: Asymptotically almost surely, 11–12, 20, 21, 26, 333, 337, 384.

AAAI: American Association for Artificial Intelligence (founded in 1979); Association for the Advancement of Artificial Intelligence (since 2007), 251.

Abaroth (= Barlow, David Stewart), 506.

Abel, Niels Henrik, 87–88.

Abío Roig, Ignasi, 633.

Absorbed clauses, 352.

Accordion solitaire, 646.

Ace Now, 8, 19.

Achlioptas, Dimitris (), 585.

Ackerman, Eyal (), 523.

ACT(c), 258, 309.

ACT(k), 250–252, 259–260, 309, 316.

Acta Mathematica, 87–88.

Active elements of a list, 40.

Active list of items, 67.

Active path, 197.

Active ring, 216, 573.

Activity scores, 150, 251, 258–260, 309, 316, 339, 603.

Acyclic digraphs, 473, 667.

Acyclic orientation, 345.

Adams, Douglas Noel (42), 310.

Adaptive control, 230, 310.

Addition, encoding of, 284–285, 298; see also Full adders, Half adders.

Adjacency matrix, 645.

Adjacent pairs of letters, avoiding, 612.

Adler, Oskar Samuel, 431.

Affinity scores, 144.

Agarwal, Akshay Kumar (), 399.

Agility level, 260, 308, 342.

Agility threshold (ψ), 260–261, 308–311, 604, 650.

AGILITY variable, 260, 342, 604.

Agriculture, 78.

Ahearn, Stephen Thomas, 460.

Ahlswede, Rudolph, 379.

Ahmed, Tanbir (), 189, 331.

Ahrens, Wilhelm Ernst Martin Georg, 34, 55, 399, 462.

Ainley, Eric Stephen, 504, 647.

Alava, Mikko Juhani, 264.

Aldous, David John, 378, 583.

Alekhnovich, Michael (Misha) Valentinovich (), 54.

Algorithm L^0, 223, 331.

Algorithms for exact covering, 67–70, 88–90, 95–98.

modifications to, 126, 127, 132, 133, 138, 183, 422, 442, 445, 542.

with minimum cost, 112–113, 116–118.

without backtracking, 127–128, 149.

Alhambra palace, 501.

Alice, 204–208, 323–325.

All-different constraint, 355, 438, see At-most-one constraint.

All-interval tone row, 135.

All solutions, 327–328, 630.

Almost sure events, 11, see a.s.

Alon, Noga Mordechai (), 358, 388–389, 444, 618, 624.

Aloul, Fadi Ahmed (), 296, 645, 648.

Alphabet, 129.

Alphabet blocks, 59, 136.

AMM: The American Mathematical Monthly, published by the Mathematical Association of America since 1894.

Anacrostic puzzle, 62.

Analysis of algorithms, 9–10, 21, 22, 28, 57, 58, 98–103, 120–121, 149, 154, 330–336, 342–344, 348, 406.

Ancestors, 227.

nearest common, 669.

AND operation, 193, 194, 197.

bitwise (x & y), 17, 128, 212, 213, 215, 221, 222, 250, 252, 260, 265, 344, 400, 419, 542, 560, 573–575, 584, 605.

André, Pascal, 315.

Anisimov, Anatoly Vasilievich (), 613.

Annexstein, Fred Saul, 637.

Answers to the puzzles in the answers, 671–673.

Anthracene, 162, 169.

Anti-maximal-element clauses, 240, 246, 281, 299, 337, 339, 341, 351.

Anti-wave, 449.

Antisymmetry, 362.

Aperiodic words, 38, 57, 404.

Appier dit Hanzelet, Jean, 241.

April Fool, 191.

ARCS(v) (first arc of vertex v), 62, 414.

Arctic circle, 482.

Ardila Mantilla, Federico, 620.

Aristophanes of Athens, son of Philippus (), xiii.

Arithmetic-geometric mean inequality, 28, 375, 383, 515.

Arithmetic overflow, 113.

Arithmetic progressions, 188, 298.

avoiding, 319; see also waerden (j, k; n).

Armbruster, Franz Owen, 52.

Armies of queens, 364.

Aromatic hydrocarbons, 162.

Array, 3-dimensional, 456.

Articulation points, 116, 182, 545.

Asakly, Walaa (), 469.

Asao, Yoshihiko (), 529.

Ascents of a permutation, 394, 522.

Ase, Mitsuhiro (), 538.

Asín Achá, Roberto Javier, 631.

Aspects of tiles, 485.

Assembly language, 82.

Asserting clause, see Forcing clause.

Assignment problem, 668.

Assignments, 185, 214.

Associative block design, 188.

Associative law, 149, 591.

Asymmetric Boolean functions, 362.

Asymmetric elimination, 624.

Asymmetric solutions, 420, 474, 491.

Asymmetric tautology, see Certifiable clauses.

Asymptotic methods, 11, 12, 16, 26, 55, 147–148, 237–238, 331–335, 348, 418, 423, 424, 574, 580, 582, 590, 594, 647.

Asterisk (*), vii.

At-least-one constraint, 355, 629.

At-most-one constraint, 190, 281–283, 287, 288, 304, 318, 333, 354, 355, 602, 629, 630, 653.

Atomic events, 1.

ATPG: Automatic test pattern generation, see Fault testing.

Atserias, Albert Perí, 626.

Aubrey, John, 416.

Audemard, Gilles, 256.

Aurifeuille, Léon François Antoine, factors, 198.

Autarkies, 228, 255, 330, 336, 361, 578, 579, 581.

testing for, 330, 578.

Autarky principle, 228.

Automatic test pattern generation, see Fault testing.

Automaton, 359, 636, see also Cellular automata.

Automorphisms, 137, 292, 295, 364, 426, 427, 430, 451–453, 456–458, 463, 528, 561, 600, 641; see also Symmetry breaking.

Autosifting, 584.

Auxiliary variables, 190, 192, 199, 201, 244, 281, 285, 288, 289, 293, 319, 320, 332, 354–358, 550, 626, 632, 640–643, 644–645, 651.

AVAIL stack, 621.

Average-case bounds, 28, see Analysis of algorithms.

Averages, 28, 304, see Expected value.

Avoiding submatrices, 290–291.

Awkward trees, 591.

Axial symmetry, 174, 428–429, 495, 528.

Axiom clauses, 238, 243, 284, 628, 630.

Aztec diamonds, 155, 157, 482.

Azuma, Kazuoki (), 9–10, 20.

B(p_1,...,pm), see Multivariate Bernoulli distribution.

Bm,n(p), see Cumulative binomial distribution.

Bn(p), see Binomial distribution.

Babbage, Charles, 56.

Bacchus, Fahiem, 257, 635.

Bach, Johann Sebastian, 63.

Backjumping, 54, 248, 252, 258, 316, 597, 600, 603.

Backmarking, 54.

Backtrack programming, 30–65, 69, 90, 97–98, 117–118, 144, 188, 211–218, 222–223, 248, 278, 289, 312, 313, 316, 335, 360, 424, 430, 443, 460–461, 523, 532, 554, 568, 583, 595, 600, 649.

efficiency of, 398.

history of, 30, 33–34, 53–54.

introduction to, 30–64.

variant structure, 54, 414.

Backtrack trees, 31, 32, 35, 37–39, 46–48, 52, 54, 55, 73, 98–100, 104–107, 126, 328, 406, 407, 411, 434, see also Search trees.

estimating the size of, 48–49, 58–59.

Backward versus forward, 21, 124.

Bailleux, Olivier, 192, 210, 319, 321, 327, 358, 636.

Baillie, Andrew Welcome Spencer, 511.

Baker, Andrew Baer, 282.

Balanced coloring, 126.

Balanced masyu solutions, 540.

Balas (Blatt), Egon, 54, 570.

Baldassi, Carlo, 277.

Ball, Walter William Rouse, 364.

Ball-piling, 168–169.

Ballot numbers, 262, 383.

Balls and urns, 6–7, 18–20, 382, 585.

Banbara, Mutsunori (), 628, 631, 632.

Banderier, Cyril, 396.

Barequet, Gill (), 523.

Barlow, David Stewart (= Abaroth), 506.

Barnes, Frank William, 514.

Barris, Harry, iv, 65.

Bartley, William Warren, III, 313.

Barwell, Brian Robert, 505.

Barycentric coordinates, 168–169, 446–447, 458.

Base placements, 161–162, 164, 169, 484–485, 497, 505, 518, 519, 528.

Basis theorem for packing, 173, 526.

Basket weavers, 325.

Batcher, Kenneth Edward, 630.

Baumert, Leonard Daniel, 54, 413, 629, 691.

Baxter, Glen Earl, 395.

permutations, 27, 522.

Baxter, Nicholas Edward, 170.

Bayardo, Roberto Javier, Jr., 316.

Bayes, Thomas, 14.

networks, 279.

BCP: Boolean constraint propagation, see Unit propagation.

BDD: A reduced, ordered binary decision diagram, 5, 201–202, 286, 287, 316, 321, 332, 358, 365, 378, 528, 541, 544, 552, 557, 558, 561, 566, 584.

BDD base, 583.

Bean, Richard, 531.

Beauty contest, 138, 152, 156, 158, 162, 546.

Becker, Joseph D., 517.

Beeler, Michael David, 500.

Bees, queen, 55, 424.

Behrens, Walter Ulrich, 78, 79, 129.

Beier, René, 396.

Belief propagation, 279.

Bell, Eric Temple, numbers ϖn, 15, 101–102, 147–148, 467, 530.

Bell, George Irving, III, 516–518.

Bellman, Richard Earnest, xi.

Beluhov, Nikolai Ivanov (), ix, 146, 177, 178, 465, 466, 482, 535, 537, 538, 542, 544, 545, 547, 548.

Ben-Dor, Amir (), 653.

Ben-Sasson, Eli (), 241–242, 337, 595.

Benchmarks, 81, 219, 315–317, 323, 331, 490, 554, 570, 631.

100 test cases, vii, 297–308, 311, 366, 368.

Bender, Edward Anton, 497, 614.

Benedek, György Mihály Pál (= George Mihaly Pal = ), 534.

Benjamin, Herbert Daniel, 157, 159, 494, 505.

Bennett, Frank Ernest, 432.

Bent tricubes, 166, 528.

Bent trominoes, 79, 82, 509.

Benzene, 162.

Beresin, May, 639.

Berg, Alban Maria Johannes, 135.

Berghammer, Rudolf, 568.

BerkMin solver, 316.

Berlekamp, Elwyn Ralph, 86, 201, 506.

Berman, Piotr, 588.

Bernhart, Frank Reiff, 552.

Bernoulli, Daniel, 380.

Bernoulli, Jacques (= Jakob = James), 53.

distribution, multivariate, 14, 18, 20, 273.

numbers, 658, 663.

Bernoulli, Nicolas (= Nikolaus), 380.

Berthier, Denis, 425.

Bertrand, Joseph Louis François, 87–88.

Besley Tollefson, Serena Sutton, 510.

Bessel, Friedrich Wilhelm, functions, generalized, 496.

BEST table, 117.

Beta distribution, 14.

Bethe, Hans Albrecht, 279.

Better reasons, 341.

Bezzel, Max Friedrich Wilhelm, 53.

Bhatia, Rajendra (), 372.

Bias messages, 276.

Biased random bits, 196, 605.

Biased random walks, 59, 407.

Biaxial symmetry, 174, 465, 528, 546.

Bible verse, 548.

Bicomponents, 164, 545.

Bidiagonal symmetry, 171, 174, 546.

Bienaymé, Irénée Jules, inequality, 4.

Biere, Armin, ix, 250, 260, 280, 313, 316, 350, 552, 622, 624, 625, 633, 644.

Big clauses, 329.

Bilateral symmetry, 174, see Biaxial symmetry.

BIMP tables, 220–225, 227, 229, 308, 328, 578, 599.

Bin-packing problem, 11, 20.

Binary addition, 298.

Binary Arts, 460.

Binary clauses, 187, 190, 220, 308, 318, 339–340.

Binary constraints, 134, 355.

Binary decoder, 363.

Binary implication graph, see Dependency digraph, 225.

Binary matrices, see Matrices of 0s and 1s.

Binary multiplication, 192.

Binary notation, 14, 126.

Binary number system, 193, 282.

Binary operators, 105, 132, 149–150, 424, 528.

Binary partitions, 59.

Binary random variables, 2, 3, 5, 13–15, 20, 25, 27.

Binary recurrence relations, 553.

Binary relations, 240.

Binary search, 435, 551.

Binary search trees, 24, 124, 424.

Binary strings, 365.

Binary tensor contingency problem, 326, 335.

Binary trees, 172.

Binary vectors, 3, 9, 13–14, 25, 108, 532.

Binet, Jacques Philippe Marie, 29, 391.

Bing, R. H., 370.

Bingo, 12–13.

Binomial coefficients, 333, 394.

Binomial convolutions, 25, 614.

Binomial distribution, 14, 24, 377, 392, 396.

cumulative, 14–15, 375, 406.

Binomial trees, 49.

Bipair: Two pairs of options that cover the same items, 107–108, 119, 150–151, 157, 473, 485, 492, 499.

Bipartite graphs, 107, 128, 242, 361, 416, 470, 654.

Bipartite matching problems, 103, 104, 128, 134, 149, 154, 155, 334, 473, 547, 668.

Bipartite structure, 274.

Biquadruples, 472.

Birthday greeting, 100.

Birthday paradox, 233.

Bishop moves, 145–146, 325.

Bit vectors, 3, 9, 13–14, 25, 108, 532.

Bitland, 131.

Bitmaps, 181, 201, 323.

Bitner, James Richard, 34, 54, 526.

Bitriples, 150, 472.

Bits of information, 24.

Bittencourt Vidigal Leitão, Ricardo, 438, 439, 443.

Bitwise operations, 33, 55, 72, 76, 127, 144, 195, 196, 265, 342, 345, 410, 605, 610, 622–623.

AND (&), see AND operation.

median (〈xyz 〉), 545.

OR (|), see OR operation.

XOR (⊕), see XOR operation.

Björklund, John Nils Andreas, 149, 471.

Björner, Anders, 379.

Black and blue principle, 330, 580.

Black and white cells, 85–86, see Parity of cells.

Black and white principle, 330.

Blackwell, David Harold, 381.

Blair, Eric Arthur [= Orwell, George], 183.

Blake, Archie, 314.

Blecher, Aubrey, 469.

blit, 598, 600.

Bloch, Cecil Joseph, 519.

Block codes, 37, 56.

Block decomposition, 639.

Block designs, 290.

Block diagonal matrices, 361.

Blocked clauses, 286, 579, 624, 625, 633.

binary, 330.

elimination of, 351.

Blocked items, 109.

Blocked self-subsumption, 351.

Blocking digraph, 579.

Blocks in kakuro, 180–182.

Blocks in Life, 561, 564.

Bloom, Burton Howard, coding, 622.

Bloom, Thomas Frederick, 549.

Bob, 204–208, 299, 323–325.

Boddington, Paul Stephen, 455.

Body-centered cubic lattice, 518.

Böhm, Max Joachim, 315.

Bollobás, Béla, 238, 388, 584.

Bonacina, Maria Paola, 313.

book graphs, 310.

Boole, George, 313.

Boolean chains, 193, 195, 196, 286, 298, 357, 667.

optimum, 362–363.

Boolean formulas, 185.

Boolean functions, 5, 15, 198–200, 378, 380, 528.

dual of, 378.

expressible in kCNF, 584.

monotone, 5, 380.

symmetric, 16.

synthesis of, 362–363.

Boolean programming problem, 667.

Boolean random variables, see Binary random variables.

Boolean vectors, see Bit vectors.

Boothroyd, Michael Roger, 451.

Boppana, Ravi Babu, 358.

Borel, Émile Félix Édouard Justin, 87–88.

Borgs, Christian, 238.

Borodin, Allan Bertram, 54.

Botermans, Jacobus (= Jack) Petrus Hermana, 410.

Bottleneck optima, see Minimax solutions.

Bottom-up algorithms, 616.

Boufkhad, Yacine (), 192, 210, 315, 319, 321, 327, 358, 636.

BOUND field, 97–98, 145.

Boundary markers, 57.

Boundary sets, 242–243, 338, 364, see also Frontiers.

Boundary variables, 594.

Bounded model checking, 200–208, 316, 321–325, 341, 363–364.

Bounded permutation problem, 103, 148–149, 154.

Bounding box, 130.

Bounds and rooms, 171–172, 394.

Bounds in futoshiki, 174.

Bousquet-Mélou, Mireille Françoise, 60, 430.

Boutillier, Cédric Grégory Marc, 483.

Bouwkamp, Christoffel Jacob, 485, 511, 513.

Bower, Richard John, 514.

Boxes in sudoku, 74, 78, 129.

Boyce, William Martin, 395.

Boyd, Stephen Poythress, 377, 399.

Boyer, Christian, 424.

Bracket notation, 2.

Bracketing property, 378.

Bradley, Milton, 452.

Branch, choice of, see MRV heuristic, Nonsharp preference heuristic, Sharp preference heuristic.

Branching heuristics, 289, 328, see also Decision literals.

Branching programs, 119, 122, 286, 357, 358.

Branchless computation, 606.

Braunstein, Alfredo, 274, 275, 620.

Breadth-first search, 127–128, 221, 227, 252, 314, 399, 599.

Break count, 263.

Breaking symmetry, see Symmetry breaking.

Brennan, Charlotte Alix, 469.

Bricks, 82, 143, 168.

Briggs, Preston, 41.

British National Corpus, 36, 413.

Broadcasting, 354.

Broadword computations, 195, 196, 342, 345, 610, 622, 667.

Broder, Andrei Zary, 387, 388.

Broken diagonals, see Wraparound.

Brotchie, Alastair, 437.

Brouwer, Andries Evert, 430.

Brown, Cynthia Ann Blocher, 214, 216, 315, 335, 590.

Brown, John O’Connor, 370.

Brown, Thomas Craig, 549.

Bruijn, Nicolaas Govert de, 82, 123, 168, 526.

cycles, 91, 134, 155, 670.

Brummayer, Robert Daniel, 633.

Brunetti, Sara, 570.

Bryant, Randal Everitt, 191, 551.

BST(l) field, 575.

BSTAMP counter, 575.

Bucket elimination, see Frontiers.

Buckingham, David John, 561, 564.

Buddy system, 220, 328, 599.

Bugrara, Khaled Mohamed (), 590.

Bugs, 200, 253, 261, 317, 604.

Buhler, Joe Peter, 370.

Bulnes-Rozas, Juan Bautista, 579.

Bumped processes, 205–206, 324, 566.

Bumping the current stamp, 44, 58.

Bunch, Steve Raymond, 34.

Bundala, Daniel, 560.

Bundgård, Thorleif, 507.

Buresh-Oppenheim, Joshua, 54.

Burns, James Edward, 568.

Burnside, William Snow, lemma, 450, 465.

Buro, Michael, 315.

Buss, Samuel Rudolph, 337, 634.

Bystanders, see Easy clauses.

C-SAT solver, 315.

Cache-friendly data structures, 39.

Cache hits, 120.

Cache memories, 208.

CACM: Communications of the ACM, a publication of the Association for Computing Machinery since 1958.

Cages in kenken puzzles, 175–176.

Calabro, Christopher Matthew, 652.

California Institute of Technology (Caltech), 432.

Candidate variables, 224–228, 315, 578.

Cannonballs, 168.

Canonical arrangements, 62, 83, 126, 426, 455.

bipairs, 108, 150–151, 473.

bricks, 168.

Canonical forms, 322, 612.

Cantelli, Francesco Paolo, inequality, 377, 395.

Cantor, Georg Ferdinand Ludwig Philipp, 87–88.

Cardinality constraints, 191–192, 210, 288, 290, 297, 298, 305, 319, 327, 551, 552, 557, 558, 560, 568, 636, 648, 649.

for intervals, 284, 554, 644.

Carlier, Jacques, 315.

Carlitz, Leonard, 346.

Carlsen, Ingwer, 482.

Carlson, Noble Donald, 507.

Carriers in Life, 561, 564.

Carroll, Lewis (= Dodgson, Charles Lutwidge), v, 313–314.

Carry bits, 193, 196, 285, 556, 557.

Carteblanche, Filet de (pseudonym, most likely of C. A. B. Smith), 52.

Cartesian coordinates, 80–85, 142, 446.

Cartier, Pierre Emile, 267, 270, 347.

Casanova de Seingalt, Giacomo Girolamo, 380.

Case analysis, 211, 314.

Castawords, 444.

Castles, 159.

Catalan, Eugène Charles, 87–88.

numbers, 380, 416, 497.

Catel, Peter Friedrich, 487.

Cauchy, Augustin Louis, 390–391, 393.

distribution, 26.

Cavanaugh, James, iv.

Cavenagh, Nicholas John, 399.

Cayley, Arthur, 59, 418.

CDCL (conflict driven clause learning) solvers, 246–255, 287, 305, 316–317, 339.

combined with lookahead solvers, 313.

compared to lookahead solvers, 282–284, 302–305, 366, 654.

Cells of memory, 39, 212, 306–308.

Cells versus pieces, 485.

Cellular automata, 201, 566.

Census data, 115.

Central (180°) symmetry, 126, 143, 171, 174, 428–429, 450, 454, 464–465, 490, 495, 528.

Certifiable clauses, 352, 624.

Certificates of unsatisfiability, 253–255, 341, 353, 360, 362.

Chaff solver, 251, 316.

Chain rule for conditional probability, 14, 372, 618.

Chains, 640, see also Boolean chains, Resolution chains, s-chains.

Channel assignment, 320.

Channeling clauses, 628.

Characteristic function, 26.

Characteristic polynomial of a matrix, 347, 582.

Charikar, Moses Samson (), 387.

Chatterjee, Sourav (), 50, 391.

Chavas, Joël, 275.

Chayes, Jennifer Tour, 238.

Chebyshev (= Tschebyscheff), Pafnutii Lvovich (), 377, 388.

inequality, 4, 9, 16, 394, 585.

monotonic inequality, 379.

polynomials, 371, 386, 611.

Checkerboard coloring, 85–86, see Parity of cells.

Chemistry, 162.

Chen, Hongyu (), 523.

Cheng, Chung-Kuan (), 523.

Chervonenkis, Alexey Yakovlevich (), 27.

Cheshire Tom, 208–210, 299, 326–327.

Chess, 191, 354.

Chessboards, 30–34, 50–54, 55, 59, 84, 93–94, 145–146, 155, 156, 202, 209, 283, 290, 299, 322, 364.

Chesterton, Gilbert Keith, 370.

Chicks, eggs, and hens, 15.

Chiral pairs, 82, 91, 167, 169, 449, 455, 511, 518.

Choice of item to cover, see MRV heuristic, Nonsharp preference heuristic, Sharp preference heuristic.

Chordal graphs, 347–348.

Christmas, 92, 144, 159.

Christofides, Demetres (), 379.

Chromatic number χ(G), 137, 283, 319–320, 331, 358, 645.

Chung Graham, Fan Rong King (), 395.

Chuzzlewit, Martin, 548.

Chvátal, Václav (= Vašek), 189, 236, 243, 549.

Cimatti, Alessandro, 316.

Circle, discrete, 158.

Circle ratio (π), see Pi.

Circuits, Boolean, 194, 285–287, 298, see also Boolean chains.

Circular lists, 216.

Circular table, 125, 154, 471.

Ciucu, Mihai Adrian, 482–483.

Civario, Gilles, 76.

Clarke, Andrew Leslie, 506.

Clarke, Arthur Charles, 156.

Clarke, Edmund Melson, Jr., 316.

Clarkson, James Andrew, 397.

Clashing pairs of letters, 268.

Clausal proofs, see Certificates of unsatisfiability.

Clause: A disjunction of literals, 186–187, 667–668.

Clause activity scores, 258, 603.

Clause-learning algorithms, 245–246, 287, 302, 305, 316–317, 338–339.

Clauses per literal, 334, 595; see also Density of clauses.

Claw graph, 613.

Claw tetracube, 82.

Cleansings, 434.

Clichés, 260.

Clique dominators, 144.

Clique hints, 284, 298, 355.

Clique Local Lemma, 349.

Cliques, 16–17, 137, 265, 284, 318, 346, 351, 353, 355, 363, 495, 554, 668.

covering by, 349.

Close packing of spheres, 168–169.

Closed lists, 42–43.

Closed paths, 177.

Closest strings, 298, 365, 366.

Clueless anacrostic, 62.

Clueless jigsaw sudoku, 130.

Clusters, 350.

CMath: Concrete Mathematics, a book by R. L. Graham, D. E. Knuth, and O. Patashnik.

CNF: Conjunctive normal form, 193, 285, 338, 357, 405, 557, 560.

Coalescing random walk, 21.

Cocomparability graphs, 613, 614.

Codes for difficulty of exercises, xi–xiii.

Codewords, commafree, 37–46, 57–59.

Coe, Timothy Vance, 565.

Coexisting armies of queens, 364.

Coffin, Stewart Temple, 516.

Cographs, 347, 614.

Cohen, Bram, 263, 610.

Cohn, Henry Lee, 482, 483.

Coil-in-the-box: A snake-in-the-box cycle, 146, 161, 465.

Coin tosses, 11–12, 19, 20, 58, 392.

Coja-Oghlan, Amin, 585.

Colexicographic order, 56, 570, 642.

Collins, Stanley John “Alfie”, 163.

Colon notation for colors, 88.

Color controls for exact covering, 87–91, 122–123, see XCC problem.

for MCC problems, 94–95.

COLOR field, 88–90, 109, 433, 434, 446, 480–481.

Color patches, 139.

Color symmetries, 91.

Colored cubes, 91, 142–143.

Coloring a graph, 137, 157, 190–191, 283–284, 337, 363, 495, 624, 668.

fractional, 319–320.

multiple, 319.

of queens, 126, 283–284, 298–299, 355.

radio, 320.

Coloring arguments, 84.

Column sums, 22, 335.

Columns as “items”, 66, 123.

Columnwise ordering, 482.

Columnwise symmetry, 290, 365.

Combinations, generation of, 55, 419.

Combinations for kakuro, 181.

Combinatorial nullstellensatz, 23.

Commafree codes, 37–46, 54, 57–59.

commit(p, j), 90, 120, 446.

commit′(p, j), 474.

Commutative law, 132, 211, 364, 385, 591.

partial, 267, 614–615.

Comparator modules, 299, 321.

Comparison, lexicographic, 285, 295–297.

Comparison of running times, 218–219, 223, 253, 281–284, 289–291, 294, 296, 302–312, 366, 368, 582, 601, 628, 645, 654.

Compensation resolvents, 223, 328, 331.

Competitions, 315–317, 655.

Compilers, 43.

Complement of a graph, 318.

Complement under central symmetry, 171.

Complementation of unary representations, 284.

Complemented literals, 186–188, 221, 246–248, 262, 295, 574, 630.

Complete binary trees, 192, 319, 594.

Complete bipartite graphs Km,n, 107, 360, 470, 614, 618.

Complete graphs Kn, 102, 108, 120–121, 148, 151, 154, 268, 335, 337, 362, 481, 550, 626.

Complete k-partite graphs, 614, 626.

Complete t-ary trees, 344.

Completion ratio, 73, 126.

Components, see Connected components.

Compositions, 381.

Compressed tries, 401–402.

Compressing, see Purging unhelpful clauses.

Concatenated shapes, 173.

Concatenated strings, 37.

Concave functions, 4, 377, 396.

Conditional autarkies, 579.

Conditional distribution, 3, 389.

Conditional expectation, 2–3, 15–19.

inequality, 5, 16, 334, 378.

Conditional probability, 1–2, 13–14, 380.

Conditional symmetries, 291, see Endomorphisms.

Conditioning operations (F | l and F | L), 211, 280, 327, 341, see Unit conditioning.

Cones in trace theory, 271.

Confidence level (ψ), 277, 619.

Conflict clauses, 247, 254, 355; see also Preclusion clauses.

Conflict driven clause learning, 246–253, 287, 305, 316–317, 339.

Conflict graph, 495.

Conflicts, 246, 308, 316.

Congruent pairs, 162.

Conjugate partitions, 148.

Conjugate subgroups, 528.

Conjunctive normal form, 185, 193, 285, 338, 357, 557, 560.

irredundant, 621.

Conjunctive prime form, 288.

Connected components, 136, 144–145, 153, 164, 169, 436, 440, 442, 478, 488, 586.

Connected graphs, 361.

Connected subsets, 62.

Connectedness testing, 353–354.

Connection puzzles, 298, 354.

Connelly, Robert, Jr., 382.

CoNP-complete problems, 187, 571.

Consecutive 1s, 272, 359, 618.

Consensus of implicants, 314.

Consistent Boolean formulas, see Satisfiable formulas.

Consistent partial assignments, 214, 349.

Constants, fundamental, 656–658.

Constrained variables in partial assignments, 349–350.

Constraint satisfaction problems, 92, 134, 413.

Contact system for adjacent tiles, 449.

Contention resolution, 25–26.

Contests, 315–317, 655.

Context free languages, 359.

Contiguous United States of America, 114–116, 118, 153, 320.

Contingency tables, binary, 326.

3D, 335.

Continued fractions, 460, 496.

Contour integration, 26.

Convex combinations, 377, 393.

Convex functions, 4, 8, 16, 20, 27, 377, 383, 397, 580.

strictly, 390.

Convex hulls, 611.

Convex polygons, 141–142, 163, 499–500.

in triangular grids (simplex ), 141–143, 155, 169, 483, 516.

Convex polyominoes, 130, 131.

Convolution of sequences, 25, 414.

Convolution principle, 614.

Conway, John Horton, 80, 86, 139, 156, 161, 201, 323, 485, 506, 508, 565.

Cook, Matthew Makonnen, 409.

Cook, Stephen Arthur, 245, 246, 314–315, 338, 593, 601.

cook clauses, 341.

Cooper, Alec Steven, 649.

Coordinate systems for representation, 451.

barycentric, 168–169, 446–447, 458.

Cartesian, 80–85, 142, 446.

even/odd, 178, 447, 450–451, 455, 456, 497, 502, 504–505, 510, 513.

octahedron, 449.

row/column, 70–72, 74–75.

triangular grid, 138, 163.

Copyrights, iv, 543.

Core assignments, 350.

Core of Horn clauses, 358, 580.

Coriand, Michael Johannes Heinrich, 556.

Corner-to-corner paths, 50–51, 59, 63.

Correlated random variables, 17–18, 372, 381.

Correlation inequalities, 17.

COST, 117–118, 479–480.

Costs, 47, 111–118, 123, 152–154, 407.

Coupling, 22–23.

from the past, 385.

Coupon collecting, 21, 392, 584.

Covariance, 2, 14, 17, 373, 380.

Cover, Thomas Merrill, 13, 372.

cover(i), 68.

cover′(i), 90, 117, 120.

cover″(i), 109.

Covering all points, 23.

Covering an item, 67, 109.

Covering assignments, 350, 585, 619.

Covering problems, 94, 153, 186, 443, 557, 558, 668; see also Exact cover problem, Tilings by dominoes.

Covering strings, 365.

CPLEX system, 210, 653.

CPU: Central Processing Unit (one computer thread), 305.

Crawford, James Melton, Jr., 282, 297.

Cray 2 computer, 321.

Crelle: Journal für die reine und angewandte Mathematik, an international journal founded by A. L. Crelle in 1826.

Crick, Francis Harry Compton, 37.

Crisscross puzzles, composing, see Wordcross puzzles.

Critical sections, 205–207, 324–325.

Cross of polycubes, 167.

Cross Sums puzzles, 181.

Crossings, 459.

Crossover point, see Threshold of satisfiability.

Crossroads, 157.

Crossword puzzle diagrams, 136.

Crossword puzzles, 180.

Crusoe (= Kreutznaer), Robinson, xiv.

CSP: The constraint satisfaction problem, 92, 134, 647.

CTH field, 446.

Cube and conquer method, 313.

Cube Diabolique, 166.

Cubes, 52–53, 59, 139, 142–143.

coordinates for, 85, 142.

numbers of the form n^3, 92.

wrapped, 157.

Cubic graphs (3-regular, trivalent), 331, 338, 595.

Cubie: A 1×1×1 cube inside a larger box, 82.

Cuboids, 82, 142, 510, 511.

Cufflink pattern, 619.

Culver, Clayton Lee, 549.

Cumulative binomial distribution, 14–15, 375, 406.

Cut rule, 243.

Cutler, William Henry, 460, 523, 527.

CUTOFF, 433–434.

Cutoff parameters, 225, 329.

Cutoff principle, 35.

Cutoff properties, 30, 33, 38, 46, 55.

Cutoff threshold, 117, 153.

Cutsets, 536.

Cutting planes, 368, 570.

Cycle detection problem, 624.

Cycle graphs Cn, x, 22, 319, 320, 344, 386, 555, 626; see also Loops.

Cycle structure of a permutation, 149, 292, 296–297, 347, 362, 641.

Cyclic DPLL algorithm, 217.

Cyclic patterns, 203.

Cyclic permutations, 140, 347.

Cyclic shifts, 38, 57.

Cylindrical tilings, 454.

d+(v) (out-degree of v), 410, 439, 529.

da Vinci, Leonardo di ser Piero, 191.

Dadda, Luigi, 193, 298, 320, 357.

Dags: Directed acyclic graphs, 155, 488, 667.

of resolutions, 238–240, 254.

Daily puzzle, 158.

Dainarism, 529.

Damerow, Valentina, 396.

Damping factors, 150, 230, 251, 258, 260, 277–278, 309, 310, 339.

Dancing links, v–vi, 35, 65–183, 189, 318, 572, 652, 655.

sometimes slow, 434–435, 439, 460, 477, 493, 513, 538.

Dancing slitherlinks, 536.

Dancing with ZDDs, 119–123.

Daniel, Samuel, 369.

Dantchev, Stefan Stoyanov (), 294.

Darrah, William, 460.

Darwiche, Adnan Youssef (), 251, 626.

Darwin, Charles Robert, 29.

Data streams, 393.

Data structures, 32–34, 37, 39–42, 46, 58, 65–69, 96–97, 109, 120, 172, 212–218, 220–222, 227, 250–251, 264, 279–280, 327–329, 339–340, 343, 351, 602, 637.

Database, shared, 25–26.

Davenport, Harold, 391.

Davis, Horace Chandler, 372.

Davis, Martin David, 193, 215–216, 314, 684.

Dawson, Thomas Rayner, 354, 465, 505.

Daykin, David Edward, 379.

de Bruijn, Nicolaas Govert, 82, 123, 168, 526.

cycles, 91, 134, 155, 670.

de Carteblanche, Filet (pseudonym, most likely of C. A. B. Smith), 52.

de Jaenisch, Carl Ferdinand Andreevitch (), 93, 398.

de La Vallée Poussin, Charles Jean Gustave Nicolas, 388.

de Moivre, Abraham, 382.

martingale, 19.

de Montmort, Pierre Rémond, 380.

De Morgan, Augustus, 56, 401, 402.

laws, 187, 630.

de Ruiter, Johan, 60, 183, 425, 544.

de Vries, Sven, 570.

de Wilde, Boris, 577.

Dead end, 50, 177.

Deadlock, 206–207.

Debugging, 253, 261.

Dechter, Rina Kahana (), 251.

Decision literals, 246, 253, 308, 316.

Decision trees, see Search trees.

Decomposable matrices, 361.

Default parameters, 277, 309–310.

Default values of gates, 195.

Definite Horn clauses, 358.

Defoe, Daniel (= Daniel Foe), xiv.

Degenerate trees, 416.

Degree of a multivariate polynomial, 23.

Degree of a node, 47, 105, 150.

Degree of a vertex, 24, 555.

Degree sequences, 164.

Degrees of truth, 221–223, 226–227, 229–230, 580.

Dekker, Theodorus Jozef, 324.

Dekking, Frederik Michel, 423.

Del Lungo, Alberto, 570.

Delannoy, Henri-Auguste, 431.

Delayer, 239–240, 336–337.

Delest, Marie-Pierre, 497.

Deletion from a heap, 598.

Deletion operation, 35, 40–41, 404–405.

and undeletion, 65, 124.

Dell Precision 3600 workstation, xi, 714.

Delta sequence, 654.

Demaine, Erik Dylan Anderson, 497.

Demaine, Martin Lester, 497.

Demenkov, Evgeny Alexandrovich (), 644.

Density, relative, 24.

Density of clauses: The number of clauses per variable, 234–235, 334, 595, 652.

Dependence graph in trace theory, 612.

Dependence of literals, 247.

Dependency digraph (of literals), 225, 315, 352, 579, 601, 624.

Dependency-directed backtracking, see Backjumping.

Dependency graph (of events), 266, 348, 349.

Dependency on a variable, 321.

Depth-first search, 53, 314, 478, see also Backtrack programming.

Dequen, Gilles Maurice Marceau, 315.

Descartes, René, coordinates, 80–85, 142, 446.

Descents of a permutation, 394, 522.

Designing puzzles, vi, 87, 136, 140, 144, 146, 160, 166, 174–183, 367.

Determinants, 169, 346, 347, 394, 615.

Deterministic algorithm, 201, 304.

Deventer, Mattijs Oskar van, 654.

Dewey, Melville (= Melvil) Louis Kossuth, notation for trees, 398.

DFAIL field, 230, 331.

Dfalse literals, 229.

Diabolical Cube, 166.

Diaconis, Persi Warren, 50, 391.

Diagonal lines (slope ±1), 23, 31, 55, 70–72, 126, 399, 429.

Diagonals of a matrix, 208–209, 325–326.

Diagram of a trace, 268.

Diameter of a graph, 506.

Diamonds, 161.

Aztec, 155, 157, 482.

tilings by, 155, 482.

Díaz Cort, José Maria (= Josep), 235.

Dice, xviii, 12, 24, 166.

Dick, William Brisbane, 364.

Dickens, Charles John Huffam, 548.

Dictionaries, 36, 56, 413.

Dicubes, 82.

Differential equations, 468.

Difficult instances of SAT, 189, 198, 210, 235, 239–243, 302–305, 337–338, 368, 554, 556, 561, 570, 644.

Digges, Leonard, xiv.

Digital tomography, 208–210, 299, 325–327, 351, 649.

Digraphs, 57, 62, 64, 238, 292, 345, 346, 410, 436, 438–439, 473, 482, 529, 627, 667, see also Blocking digraph, Dependency digraph, Implication digraph.

acyclic, 155, 488, 667.

Dihedral groups, 91, 447, 449, 527–528.

Dijkstra, Edsger Wybe, 206, 566, 568.

DIMACS: Center for Discrete Mathematics and Theoretical Computer Science, 315.

DIMACS: DIMACS Series in Discrete Mathematics and Theoretical Computer Science, inaugurated in 1990.

Dimension reduction, 393.

Dimer tilings, 431.

Ding, Jian (), 235.

Diophantine equations, 144, 450.

Dips, 57.

Direct encoding, 282, 298, 355, 550, 628, 629, 645.

Direct sum of graphs or matrices, 346, 361.

Direct sum T⊕T′ of search trees, 105, 149–150.

Directed acyclic graphs, 155, 488, 667.

Directed acyclic graphs of resolutions, 238–240, 254.

Directed graphs, see Digraphs.

Directed graphs versus undirected graphs, 63, 667.

Discarded data, 50.

Discarding the previous learned clause, 256, 340.

Disconnected shapes, 173.

Discrepancy patterns, 298, 366.

Discrete probabilities, 1.

Disjoint sets, 53, 407.

Disjoint shortest paths, 640.

Disjunctive normal forms, 198–200, 299, 314, 559, 621.

Dissection: Decomposition of one structure into substructures, 170.

Distance d(u, v) in a graph, 626.

dynamically updating, 59.

Hamming, 127, 649.

in a plane, 112.

Distinct literals, 186.

Distributed computations, 34.

Distribution function, 395.

Divergence, Kullback-Leibler, 24–25, 407.

Divergent series, 469.

Diversity of exact coverings, 127.

Divide and conquer paradigm, 52.

Division of traces, 269, 345, 614.

DLINK field, 67–69, 88–90, 95–98, 109–110, 117–118, 120, 125, 433–434, 474–475, 480–481.

DLX algorithm, viii, 123.

DNF: Disjunctive normal form, 198–200, 299, 314, 559, 621.

Döblin, Wolfgang (= Doeblin, Vincent), 387.

Dobrichev, Mladen Venkov (), 427.

Dobrushin, Roland L’vovich (), 387.

Dodecahedron, 139, 451.

rhombic, 516.

Dodeciamond, 162.

Dodgson, Charles Lutwidge (= Carroll, Lewis), v, 313–314.

Domains, 30, 55, 56.

Dominant nodes, 105, 643.

Dominating sets, 461, 668, see also 5-queens problem.

Dominoes, 79, 131, 161, 533.

tilings by, 119, 155, 294, 298, 299, 327, 361, 362.

windmill, 160–161.

Dominosa, 131.

Don’t-cares, 558, 644.

Doob, Joseph Leo, 6, 9, 383.

martingales, 9–10, 20, 27, 384.

Doomsday function D(n), 98–100, 147.

Dorian cube, 511.

Dorie, Joseph Edward, 511.

Doris® puzzle, 454.

Dot-minus operation (x−⋅y=max{0,x−y}), x, 21–22, 463.

Dot product of vectors, 20, 26.

Double clique hints, 284, 298, 355.

Double coloring, 299, 319.

Double counting, 464–465.

Double-crostics, 413.

Double factorial, see Semifactorial.

Double lookahead, 229–230, 310, 315, 650.

Double order, 578.

Double truth, 229.

Double word squares, 133, 183, 402.

Doubly linked lists, 65, 124, 212, 621, 623.

Doubly symmetric queen patterns, 152, 420–421.

Dowels, 166.

Dowler, Robert Wallace Montgomery, Box, 167.

Downdating versus updating, 32–33, 39, 43.

Downhill resolution, 280, 350.

Downhill transformations, 279.

Downloadable programs, viii, 523.

Doyle, Arthur Ignatius Conan, 64, 256.

DPLL (Davis, Putnam, Logemann, Loveland) algorithm, 216–217, 246.

with lookahead, 222, 315.

Dragon sequence, 446.

Drive Ya Nuts puzzle, 140.

DT (double truth), 229.

Dtrue literals, 229.

Dual linear programming problem, 479.

Dual of a Boolean function, 314, 358, 378.

Dual of a hypergraph, 125.

Dual of a kakuro puzzle, 543–544.

Dual of a permutation problem, 148.

Dual of a skewed pattern, 505.

Dual oriented spanning tree, 63.

Dual solutions, 34, 36, 56.

Dubois, Olivier, 315.

Dudeney, Henry Ernest, 79, 298, 402, 421, 487, 627.

Dufour, Mark, 221.

Duhamel, Jean-Marie Constant, 396.

Dull, Brutus Cyclops, 365.

Dulucq, Serge, 395, 523.

Duplicate options, 98, 145, 147, 421.

Durfee, William Pitt, square, 640.

Dworkin, Morris Joseph, 470.

Dynamic ordering, 38–39, 52, 54, 59, 411.

Dynamic programming, xi, 493.

Dynamic shortest distances, 59.

Dynamic storage allocation, 328.

Dynamical system, discrete, 200.

e (base of natural logarithms), 656–657.

as source of “random” data, 47, 183, 196, 557.

Eager data structures, 214, 220, 340.

Earl, Christopher Francis, 520.

Eastman, Willard Lawrence, 57, 58, 404.

Easy clauses, 333.

Eaters in Life, 204, 323.

Eckler, Albert Ross, Jr., 402, 441.

Edgar, Gerald Arthur, 502.

Edge-connected cubes, 515–516.

Eén, Niklas Göran, ix, 251, 280, 350, 567, 624, 632.

Efficient: Reasonably fast, 181.

Eggenberger, Florian, 6.

Ehlers, Thorsten, 560.

Eight queens problem, see 8 queens problem.

Eightfold symmetry, 174, 180, 322, 429, 489, 535, 546, 562.

Einstein, Albert, 665.

Elective items, 88.

Elegance, 219–220, 560.

Eleven blog, 426.

Elimination of clauses, 351–352; see also Purging unhelpful clauses.

Elimination of variables, 244–245, 279–281, 285, 286, 313, 314, 338–339, 350–352, 357, 358, 620–621, 623–624, 634, 635.

Elkies, Noam David, 482.

Ell tetrominoes and tetracubes, 82, 483.

Ell trominoes, see Bent trominoes.

Ellipses, 483.

Elton, John Hancock, 375.

Embedded graphs, 132, 132, 353, 626.

Emlong, Ruby Charlene Little Hall, 507.

Empilements, 268, 345, 612.

Empirical performance measurements, 306–308.

Empirical probabilities, 27.

Empirical standard deviation, 431.

Empty clause (∈), 187, 211, 549, 655.

Empty graph (Kn¯), 268, 618.

Empty list, representation of, 40, 45, 217, 574.

Empty partial assignment, 350.

Empty set (∅), 549.

Empty string (∈), 187, 269.

Encoding into clauses, 190–194, 202, 281–289, 304, 318, 354, 363, 562, 566, 655.

ternary data, 284, 325, 363.

Endless Chain puzzle, 143.

Endomorphisms, 291–295, 361–362, 365, 645, 654.

Engelhardt, Matthias Rüdiger, 34, 398.

English words, viii, 36–37, 56, 59, 62, 94–95, 133–134, 136, 145, 152, 156, 183, 401–402, 413, 434, 443, 463–464, 484.

Enneominoes, 130, see Nonominoes.

Entropy, 24, 25, 27.

relative, 24.

Enveloping series, 378.

Eppstein, David Arthur, 100, 147.

Equal sums, encoding of, 358.

Equal temperament, 135.

Equally spaced 1s, 188, 298, 319; see also waerden (j, k; n).

Equilateral triangles, 139.

coordinates for, 138, 163.

Equivalence algorithm, 539.

Equivalence classes in trace theory, 268.

Equivalence of Boolean functions, 362.

Equivalence relations, 468.

Erdős, Pál (= Paul), 265, 291, 378, 554, 645.

discrepancy patterns, 298, 363, 366.

Ernst, George Werner, 410.

Erp rules, 279–280, 350–352, 623.

Error bars, 50.

Escher, Maurits Cornelis, 501–502.

Essentially different (inequivalent), 86, 129, 137–141, 420, 442, 445, 450, 452, 456, 507; see also Symmetry breaking.

Esser, Peter Friedrich, 450, 494, 506.

Estimates of run time, 46–49, 54, 58–59, 73, 113, 133, 445, 447, 450.

Estimating the number of solutions, 49–51.

Euler, Leonhard (), 469, 529–530.

constant γ, 47, 373, 656–657.

Euler-Gompertz constant, 102, 469.

Eulerian numbers, 381.

Evaluation of Boolean functions, 321, 362–363, 558.

Even-length cycles, 641.

Even/odd coordinate systems, 178, 447, 450–451, 455, 456, 497, 502, 504–505, 510, 513.

Even-odd endomorphisms, 294, 361–362.

Even symmetry, 528.

Events, 1–3.

Every kth cost, 153–154.

Exact cover problem, v–vi, 66–72, 98, 114, 122, 123, 127, 153, 174, 186, 189–190, 212, 318, 367, 409, 456, 492, 550, 583, 589, 654, 668.

by pairs (perfect matchings), 293–294, see also Tilings by dominoes.

by triples (3DM), 318, 589, 654–655.

extreme, 101, 147, 154.

fractional, 319–320, 479.

minimum-cost, vi, 111–118, 123, 152–154, 491.

random, 127.

strict, 98, 99, 147.

uniform, 118, 127.

with colors, see XCC problem.

with multiplicities, see MCC problem.

without backtracking, 127–128, 149.

Exact (one-per-clause) satisfiability, 367.

Exchangeable random variables, 381.

Exclusion clauses, 190, 205, 283, 298, 318, 333, 337, 602, 624, 635, 653, 654.

Exclusive or, ternary, 320.

Exercises, notes on, xi-xiii.

Existential quantifiers, 244.

Expander graphs, 242, 595.

Expected value, 2–5, 14–16, 393, see also Conditional expectation.

Exploitation stack, 623.

Exploration phase of lookahead, 224, 227–228.

Exponential behavior, 104.

Exponential generating functions, 147–148, 418.

Exponential time, 328.

hypothesis, 652.

Exponentially small, 391, see also Superpolynomially small.

Extended hexadecimal digits, 73, 80, 484.

Extended resolution, 244, 255, 317, 337, 338, 352, 579.

Exterior costs, 114, 153.

Extreme distribution, 271, 273, 347.

f↑ (maximal elements of family f), 545–546.

F pentomino, 80, 486, see R pentomino.

Façades, 166.

Face-centered cubic lattice, 168, 516, 518.

Face of a planar graph, 536.

factor_fifo (m, n, z), 194, 196, 298, 368, 556.

factor_lifo (m, n, z), 194, 298, 368, 556.

factor_rand (m, n, z, s), 194, 368.

Factorial generating function, 418.

Factoring an exact cover problem, 83–86, 102, 109, 124, 137, 164, 167, 459–460, 482, 491, 492, 499, 509, 523, 527; see also Relaxation of constraints.

Factorization of a string, 38, 57.

Factorization of problems, 52–53, 59, 60, 62, 192–194, 320, 368, 414, 556.

of traces, 270, 346, 614.

Factorizations of an integer, 146.

Failed literals, 281, 351, 359, 633.

Fair sequences, 7, 10, 19, 382, 393–394.

with respect to a sequence, 7, 382.

Fairbairn, Rhys Aikens, 485.

Fallacious reasoning, 200, 648.

Fallback points, 44.

False hits, 622.

False literals preferred, 215, 217, 251, 309–311, 650.

Falstaff, John, 548.

Families of sets, 17, 124, 537.

Fanout gates, 194–198, 320.

Farhi, Sivy, 507, 512.

Fat clauses, 242.

Fault testing, 194–198, 298, 310, 320–321, 351, 624.

Faultfree rectangle decompositions, 157, 170, 172, 429, 521.

Fédou, Jean-Marc, 430.

Feedback mechanism, 230, 288.

Feige, Uriel (), 375, 392.

Feldman, Gary Michael, 450.

Feller, Willibald (= Vilim = Willy = William), 381.

Fences, 157, 175.

Fermat, Pierre de, 194.

Fernandez de la Vega, Wenceslas, 236.

Fernández Long, Hilario, 450.

Ferrers, Norman Macleod, diagrams, 160.

FGbook: Selected Papers on Fun & Games, a book by D. E. Knuth.

Fibonacci, Leonardo, of Pisa (= Leonardo filio Bonacii Pisano),
dice, 12.

martingale, 19.

numbers, 12, 121, 154, 344, 382, 579, 618, 658.

ruler function, 610.

Fichte, Johannes Klaus, 626.

Field of a variable, 275, 349.

Fields, Dorothy, iv, 65.

FIFO: first in, first out, 194.

Finite basis theorem, 173, 526.

Finite-state automata, 359, 636.

Fink, Federico [= Friedrich], 450.

Finkel, Raphael Ari, 54.

First in, first out, 194.

First moment principle, 4, 16, 237, 332, 334.

First-order logic, 243, 314.

First tweaks, 96.

Fischetti, Matteo, 422, 477, 570.

Five-letter words of English, 36–37, 56, 59, 62, 94–95, 133–134, 136, 145, 152, 156, 183, 402.

Fixed point of recursive formula, 412.

Fixed points of endomorphisms, 293–294, 361–362.

Fixed points of messages, 276, 349.

Fixed values of literals, 221, 226–230.

FKG inequality, 5, 17, 273, 380.

Flag bits, 599.

Flajolet, Philippe Patrick Michel, 418, 497.

Flammenkamp, Achim, 562.

Flat pentacubes, 167.

Fletcher, John George, 82.

Flexibility coefficients, 275.

Flickering state variables, 325.

Flipflops in Life, 322, 327.

Flipping pieces over, 81, 82, 140, 158, 449, 452, 453, 485–487, 489.

Floating point arithmetic, 275–276, 581, 603.

overflow, 251.

Floor tiling, 299, 327, 563.

Floorplans, 171–172, 394.

Flow in a network, 22.

Flower Power puzzles, 435.

Flower snarks, 253, 331, 337, 341.

Flowsnake fractal, 502.

Floyd, Robert W, 43.

Flushing literals and restarting, 252, 259–261, 308, 316, 341, 342, 353, 598, 610.

Flye Sainte-Marie, Camille, 437.

Foata, Dominique Cyprien, 267, 270, 347.

FOCS: Proceedings of the IEEE Symposia on Foundations of Computer Science (1975-), formerly called the Symposia on Switching Circuit Theory and Logic Design (1960–1965), Symposia on Switching and Automata Theory (1966–1974).

Focus of attention, 82, 104–106, 124, 150, 225, 251, 275, 316, 437, 541, 543.

Foe, Daniel (= Daniel Defoe), xiv.

Fogel, Julian, 526.

Fool’s Disk, 60.

Footprint heuristic, 643.

Forced literals, 229.

Forcing, 75, 109, 151.

Forcing clause, 246, see Unit propagation.

Forcing representations, 288–289, 358, 359, 638.

Forests, 227, 271, 347, 496.

Forgetting clauses, 352, 353, see Purging unhelpful clauses.

Formal power series, 395–396.

Fortuin, Cornelis Marius, 17.

Forward versus backward, 21.

Four bit protocol, 299.

Four Color Theorem, 191.

Four functions theorem, 17, 379.

Four-letter codewords, 37–46, 57.

Fourfold symmetry, 171, 174, 322, 402, 465, 528, 546.

Fox-Epstein, Eli, 504.

FPGA devices: Field-programmable gate arrays, 34.

Fractal, 502.

Fractional coloring number, 319–320.

Fractional exact cover problem, 319–320.

Fragment list, 536.

Frames, 39.

Frances, Moti (), 649.

Francillon, Jean Paul, 509.

Franco, John Vincent, 315, 332, 589, 637.

Franel, Jérôme, 399.

Free literals and free variables, 222, 250, 349–350.

Free ZDDs, 480.

Freeman, Jon William, 315.

Freeman, Lewis Ransome, 30.

French, Richard John, 511.

Frere, John Hookham, xiii.

Friedgut, Ehud (), 235.

Friedland, Shmuel (), 389.

Friedman, Bernard, urn, 19.

Friedman, Erich Jay, 460, 495.

Frieze, Alan Michael, 387.

Frobenius, Ferdinand Georg, 87–88.

Fröberg, Ralf Lennart, 613.

Frontiers, 424, 526, 545, 552, see also Boundary sets.

Frost, Daniel Hunter, 251.

fsnark clauses, 253, 255, 298, 331–332, 337, 341.

FT array, 96.

Fuhlendorf, Georg, 489.

Fujiyoshi, Kunihiro (), 523.

Full adders, 193, 320, 363, 556, 632, 642.

modified, 298, 644.

Full runs, 257, 342, 599.

Funk, Jacob Ewert, 181, 543.

Furtlehner, Cyril, 275.

Futoshiki, 174–175.

G4Gn: The nth “Gathering for Gardner,” a conference inaugurated in 1993.

Gμ: One gigamem (one billion memory accesses), 72, 219, 223, 281, 305, 552.

per minute, 653.

Gadgets, 318, 367, 421.

Gallager, Robert Gray, 279.

Game of Life, 201–204, 281, 298, 321–323, 327, 351.

Games, 8, 13.

Gamma function, 397, 469, 656–657, 663.

Gange, Graeme Keith, 633.

Garbage collection, 65.

García-Molina, Héctor, 157.

Garden of Eden, 323.

Gardner, Erle Stanley, 30.

Gardner, Martin, vi, 78, 83, 191, 203, 365, 428, 456, 485, 486, 489, 490, 500–502, 504, 506, 508, 523, 527, 552, 647, 687.

Garey, Michael Randolph, 11, 545, 670.

Garfinkel, Robert Shaun, 123, 431.

Garns, Howard Scott, 74.

Gaschnig, John Gary, 54.

Gates of a circuit, 194–197, 285–287, 305, 320.

Gauβ (= Gauss), Johann Friderich Carl (= Carl Friedrich), 53, 398.

GB_GATES program, 197–198.

Gebauer, Heidi Maria, 588.

Geek art, 300–301, 497, 511.

Geerinck, Theodorus, 513.

Generalization of resolution, 590.

Generalized kakuro, 181.

Generalized toruses, 501.

Generating functions, 15, 22, 24, 28, 58, 148, 149, 160, 269, 273–274, 335, 342–344, 348, 376, 378, 381, 384, 385, 392, 393, 406, 408, 414, 415, 418, 430, 481, 552, 558, 561, 583, 586, 594, 617, 648.

exponential, 147–148, 346, 418, 647.

Generation of random objects, 385.

Generic graph, 358.

Generic option, 109.

Gent, Ian Philip, 629, 647.

Gentzen, Gerhard Karl Erich, 243.

Geoffrion, Arthur Minot, 54.

Geometric distribution, 21, 24, 392, 608, 626.

Geometric mean and arithmetic mean, 28, 375, 383, 515.

Geometric sudoku, 78.

Georges, John Pericles, 556.

Georgiadis, Evangelos (), 372.

Gerdes, Paulus Pierre Joseph, 569.

Gerechte designs, 79, 129.

Gerry, Elbridge, 131.

Gerrymandering, 131.

Gessel, Ira Martin, 612.

Ghost pentominoes, 495.

Giant component, 144, 436.

Giant strong component, 236.

Gibat, Norman Edlo, 440.

Gigamem (Gμ): One billion memory accesses, 33, 219, 223, 281, 302–305, 552.

per minute, 653.

Gilat, David, 382.

Gilbert, Edgar Nelson, 439–440.

Gillen, Marcel Robert, 491.

Ginibre, Jean, 17.

Ginsberg, Matthew Leigh, 297, 316.

Gipatsi patterns, 325.

Girth of a graph, 360, 654.

Given literals (F | l or F | L), 211, 280, 327, 341; see also Unit conditioning.

Glaisher, James Whitbread Lee, 87–88.

Gliders in Life, 203, 322–323, 561, 565.

symmetry of, 564.

Global ordering, 648.

Global variables, 55, 536.

Glucose measure, see Literal block distance.

go to statements, 446.

Goerdt, Andreas, 236.

Goh, Jun Herng Gabriel (), 540.

Goldberg, Allen Terry, 315, 589.

Goldberg, Eugene Isaacovich (), 254, 316.

Golden ratio (ϕ), 12, 121, 127, 330, 331, 344, 370, 451, 615, 656–657.

as source of “random” data, 47.

Goldenberg, Mark (= Meir) Alexandrovich (), 526.

Goldman, Jay Robert, 614.

Goldstein, Michael Milan, 410.

Golomb, Solomon Wolf, 37, 54, 79, 80, 84, 157, 413, 486, 502, 629, 691.

Gompertz, Benjamin, 102, 469.

Gondran, Michel, 478, 479.

González-Arce, Teófilo Francisco, 632.

Goodger, David John, 506.

Gordon, Basil, 37, 691.

Gordon, Leonard Joseph, 441, 442, 517, 518.

Gosper, Ralph William, Jr., 204, 373, 502, 562.

Gosset, John Herbert de Paz Thorold, 421.

Gosset, William Sealy (= Student), t-distribution, 393.

Gottfried, Alan Toby, 447, 454.

Gould, Henry Wadsworth, 102.

numbers, 101–102, 147–148, 469.

Gould, Wayne, 425.

Goulden, Ian Peter, 497.

Goultiaeva, Alexandra Borisovna (), 257.

Graatsma, William Petrus Albert Roger Stephaan, 84.

Grabarchuk, Petro (= Peter) Serhiyovych (), 428, 627.

Grabarchuk, Serhiy Oleksiyovych (), 161, 428, 627.

Grabarchuk, Serhiy Serhiyovych (), 627.

Graders, 411.

Graffiti, 548.

Graham, Ronald Lewis (), 370, 395, 520, 523, 549, 680.

Gram, Jørgen Pedersen, 87–88.

GRAND TIME puzzle, 78.

Graph-based axioms, 243, 338, 362, 654.

Graph coloring problems, see Coloring a graph.

Graph embedding, 353, 626.

Graph layout, 300–302.

Graph problems, 667–669.

Graph quenching, 298, 363–364, 645.

Gravitationally stable structures, 164, 166, 509.

Gray, Frank, codes, 565–566, 646, 669.

Greed-avoidance parameter (p), 263, 278, 302, 607–608.

Greedy algorithms, 264, 320, 356, 479.

Greedy queens, 127.

Greenbaum, Steven Fine, 286.

Greene, Curtis, 640.

Grensing, Dieter, 482.

Grid graphs (Pm □ Pn), 23, 62–63, 294, 320, 335, 346–347.

list coloring of, 335.

oriented, 62.

Grid patterns, 201–204, 208–210, 321–323, 326; see also Game of Life.

rotated 45°, 157, 160, 325–326.

Gridgeman, Norman Theodore, 409.

Griffith, John Stanley, 37.

Griggs, Jerrold Robinson, 556.

Grimmett, Geoffrey Richard, 377.

Grīnbergs (= Grinberg), Emanuels Donats Frīdrihs Jänis, 582.

Gritzmann, Peter, 570.

Gropes, 132.

Grötschel, Martin, 628.

Groupoids, see Binary operators.

Groups, 432.

Grünbaum, Branko, 485, 488.

Gu, Jun (), 261.

Guéret-Jussien, Christelle, 631.

Guibert, Olivier, 395, 523.

Guilherme De Carvalho Resende, Mauricio, 199.

Gumball machine problem, 375.

GUROBI system, 653.

Guruswami, Venkatesan (), 392.

Gut, Allan, 397.

Guy, Michael John Thirian, 86, 506.

Guy, Richard Kenneth, 86, 201, 203, 291, 468, 491, 500, 506, 511.

Gwynne, Matthew Simon, 289, 634, 637.

H-grid, 503.

Haagerup, Uffe Valentin, 397.

Hackers, 563.

Hadamard, Jacques Salomon, 87–88.

transform, 515.

Hajiaghayi, MohammadTaghi (), 235.

Haken, Armin, 241, 242.

HAKMEM, 373, 500.

Hales, Alfred Washington, 370, 691.

Halevi, Shai (), 653.

Half adders, 193, 556, 632.

Hall, Marshall, Jr., 54.

Hall Emlong, Ruby Charlene Little, 507.

Halting problem, 314.

Hamadi, Youssef (), 600.

Hamaker, William, 514.

Hamilton, William Rowan,
cycles, 143, 353, 534, 540, 669.

king paths, 535.

paths, 51, 176, 368, 669.

Hammersley, John Michael, 54.

Hamming, Richard Wesley, distance, 127, 649.

Han, Hyojung (), 600.

Handscomb, David Christopher, 54.

Handwaving, 273.

Hansson, Frans, 80, 81, 159, 488, 493, 511.

Hanzelet, see Appier dit Hanzelet.

Haralambous, Yannis (), 714.

Hard clauses, 352.

Hard sudoku, 367.

Hardest sudoku puzzle, 129.

Hardy, Godfrey Harold, 397.

Harmonic numbers, fractional, 383, 659.

Harris, Robert Scott, 78, 429.

Hart, Johnson Murdoch, 523.

Hartman, Christiaan, 566.

Haselgrove, Colin Brian, 486.

Haselgrove, Jenifer Wheildon-Brown (= Leech, Jenifer), 106, 486, 490.

Hashing, 9–10, 20, 481.

Håstad, Johan Torkel, 392.

Haswell, George Henry, 452.

Haubrich, Jacob Godefridus Antonius (= Jacques), 453.

Haven, G Neil, 315.

Hawking, Stephen William, 665.

Hawkins, Harry, 489.

Hayes, Brian Paul, 75.

Head of the list, 212.

Header elements, 589.

Header nodes, 65–68, 89, 124, 416, 480.

Heads and tails puzzles, 140.

HEAP array, 251–252, 342, 597–599, 604.

Heap data structure, 251–252, 339, 578.

insertion and deletion from, 598.

Heap ordered arrays, 117.

Heaps of pieces, 267.

Heavy tails, 26, 431.

Height of a literal, 578.

Height of a trace, 269.

Height of binary trees, 59, 416.

Heilmann, Ole Jan, 615.

Hein, Piet, 82, 83, 164, 507, 517.

Helpful rounds, 353.

Henle, James Marston, 429, 532.

Hensel, Kurt Wilhelm Sebastian, 87–88.

Heptacubes, 511.

Heptiamonds, 500, 501.

Hermite, Charles, 87–88.

Hertog, Martien Ilse van, 523.

Heule, Marienus (= Marijn) Johannes Hendrikus, ix, 221, 224, 230, 255, 259, 281–282, 288, 313, 318, 331, 366, 550, 566, 577, 603, 624, 625, 627.

Heuristic scores, 274–279.

for clauses, 256–258, 309–311, 342, 603, 650.

for variables, 224–228, 245, 251, 264, 310, 329–331, 578.

Hexacubes, 511.

Hexadecimal constants, x, 657.

Hexadecimal digits, 176.

extended, 73, 80, 484.

Hexagonal close packing, 170.

Hexagons, 55, 142, 162.

coordinates for, 162.

Hexiamonds, 141, 161, 503.

Hexominoes, 79, 129, 159.

Hexotinoes, 500.

Hidato®, 176–177.

Hidden mathematicians, 87–88.

Hidden singles and pairs, 76–77, 128, 425, 529.

Hidden weighted bit function, 357.

hide(p), 69.

hide′(p), 90, 120, 154.

hide″(p), 109.

hide′″(p), 474.

hide″″(p), 481.

Hiding an option, 69, 117.

Hierarchy of hardness, 360, 362.

Hilbert, David, 87–88, 527.

Hill, Gerald Allen, 507.

Hilton, Anthony John William, 555.

Hint clauses, 284, 298, 355.

Hirsch, Edward Alekseevich (), 579.

Historical notes, 30, 33–34, 53–54, 81–82, 106, 123, 215–216, 243–244, 289, 313–317, 387–388, 396–397, 402, 418, 435, 440, 443, 450–454, 457, 459–460, 465–469, 482–483, 485, 487, 489–490, 493, 500, 505, 509–511, 514, 517–518, 526, 529, 532, 535, 538, 595, 627.

Hitori, 182–183.

Hitori covers, 183.

Hitotumatu, Hirosi (), 123.

Hitting set problem, 416.

Ho, Boon Suan (), 540.

Hoare, Charles Antony Richard, 123.

Hobby, John Douglas, 714.

Hoek Loos company, 453.

Hoeffding, Wassily (= Wassilij), 9, 15.

inequality, 9–10, 20, 394.

Hoffman, Dean Gunnar, 168, 515.

Hoffmann, Louis (= Lewis, Angelo John), 60, 459, 509, 510.

Hoggatt, Verner Emil, Jr., 395.

Hölder, Ludwig Otto, 396.

inequality, 29.

Holes in cubies, 166.

Holes in polyominoes, 157, 496.

Hollow mazes, 654.

Holmes, Thomas Sherlock Scott, 64, 256.

Holy Grail, 509.

Homer (), 183, 666.

Homogeneous puzzles, 178, 180.

Homomorphic embedding, 353, 626.

Homomorphic images, 52.

Honest representations, 289, 634.

Honeycombs, 55.

Hoory, Shlomo (), 588.

Hoos, Holger Hendrik, ix, 309–311, 317, 344.

Hopcroft, John Edward, 545.

Horizontal and vertical symmetry, 174, 465, 528, 546.

Horn, Alfred, clauses, 317, 350, 360, 580, 627.

core of, 358, 580.

renamed, 360, 627.

satisfiability, 667.

Horsley, Daniel James, 638.

Horton, Robert Elmer, numbers, 336.

Hsiang, Jieh (), 313.

Hsiung, Chuan-Chih (), 504.

Huang, Wei-Hwa (), ix, 123, 440.

Hume, David, 185.

Hunt, Warren Alva, Jr., 255, 603.

Hurwitz, Adolf, 87–88, 399.

Hutter, Frank Roman, 309, 317.

Hydes, Horace, 452.

Hypercubes, 82, 168, 465.

Hyperedges, 125.

Hypergeometric functions, 393, 395.

Hypergraphs, 125.

2-colorability, 549.

4-colorability, 137.

Hyperoctahedral symmetries, 529.

Hyperresolution, 240, 592, 621.

Hypersolid pentominoes, 515.

Hypersudoku, 130, 137.

I pentomino, 80, 486, see O pentomino.

IBM 704 computer, 413.

IBM 1620 computer, 34.

IBM System 360–75 computer, 34.

iCauchy distribution, 26.

Icosahedron, regular, 451, 501.

Idempotent elements, 132.

Identical options, 98, 145, 147, 421.

Identity elements, 132, 149.

Idle Year solitaire, 364.

IEEE Transactions, ix.

If-then-else operation (u? v: w), 265, 286, 336, 357, 583.

ILP (integer linear programming), see Integer programming problems.

ILS: Iterated local search, 309.

Impagliazzo, Russell Graham, 54, 239, 652.

Implicant: A conjunction of literals, 668.

Implication digraph, 236–237, 328.

Implicit enumeration, 54.

Importance sampling, 25, 54.

In-degree of a vertex, 436.

in situ deletion, 69, 340.

Incidence matrices, 124.

Inclusion and exclusion, 376, 378, 388, 471, 585–586, 620.

Incomparable dissections, 172–173.

Incomplete beta function, 14.

Inconsistent clauses, see Unsatisfiable formulas.

Incroci Concentrici puzzles, 435.

Indecomposable matrices, 361.

Independent events, 2.

Independent random variables, 1, 7, 9, 10, 13–15, 20, 382.

k-wise, 1, 13.

Independent sets, 145, 191, 331, 668.

Independent subproblems, 52.

Indeterminate statements, 412–413.

Induced subgraphs, 63, 114, 145, 153, 183, 265, 436, 626.

Induction proofs by machine, 208, 567.

Indyk, Piotr Jόzef, 393.

Infinite loop, 608.

Infinite mean, 381, 382.

Infinity lemma, 138.

Information, bits of, 24.

Information gained, 24–25.

Initial guess for literals, 215, 217, 250, 309–311, 650.

Initial state X_0, 200–201, 205, 208, 324, 566.

Inner loops, 398.

Inorder traversal, 172.

Inprocessing, 279, 352.

Input and output, 304.

Input states, 359.

Insertion into a heap, 598.

Insertion operation, 40.

Instant Insanity®, 52–53, 59–60, 142–143.

Instantiations, see Assignments.

Integer multilinear representation, see Reliability polynomials.

Integer partitions, 55, 59.

Integer programming problems, 54, 210, 368, 422, 477, 525, 570, 649.

Intelligent design, 201, 321.

Interactive methods, 88, 326, 440–441.

Interior costs, 153.

Interlaced roots of polynomials, 347.

Internal zeros, 25, 391.

Internet, iv, viii, xii, 302.

Intersection graphs, 268, 345.

Intersection of solutions, 445.

Interval graphs, 271, 347.

Intervals, cardinality constrained to, 284, 554, 644.

Intervals of allowed multiplicities, 93.

Intervals of the real line, x, 4, 27, 386, 661.

Invariant assertions, 207–208, 227, 299, 324, 409, 567, 580, 581, 619, 625.

Inverse lists, 40–43, 45.

Inverse permutations, 27, 40–41, 148, 296, 629.

Inverses, 381.

Inversions of a permutation, 149, 577.

Invertible puzzles, 540, 542.

Invisible nodes, 122.

Involution polynomial of a set, 347.

Involutions, signed, 296–297, 364, 641–642.

INX array, 222, 575, 650.

IP: Integer programming, 210, 368, 570, 649.

Irredundant CNF, 621.

Irreflexive relation, 240.

Irving, Robert Wylie, 335.

Isaacs, Rufus Philip, 582.

Isolated vertices, 488, 626.

Isometric projection, 155.

non-, 166.

Isomorphic binary operators, 432.

Isosceles right triangles, 163.

Isosceles triangles, 451.

Isotopic binary operators, 424, 427.

IST(l) field, 222.

ISTACK array, 222, 329.

ISTAMP counter, 221–222, 230, 329.

ITE, see If-then-else operation.

Items, v, 66–69, 88, 123; see also Secondary items.

Iterated local search, 309.

Iteration versus recursion, 55, 398.

Itoh, Toshiya (), 388.

Iwama, Kazuo (), 588.

Jabbour, Saïd (), 600, 653.

Jabbour-Hattab, Jean (), 390.

Jaccard, Paul, 387.

index, 387.

Jackson, David Martin Rhŷs, 497.

JACM: Journal of the ACM, a publication of the Association for Computing Machinery since 1954.

Jacquet, Philippe Pierre, 589.

Jaenisch, Carl Ferdinand Andreevitch de (), 93, 398.

Jagger, Michael Philip “Mick”, 185.

Jahn, Fritz, 431.

James White, Phyllis Dorothy, 11.

Janson, Carl Svante, ix, 374, 382, 384–385, 389.

Japanese arrow puzzles, 135, 410.

Järvisalo, Matti Juhani, 289, 316, 624, 625.

Jeannicot, Serge, 579.

Jelliss, George Peter, 505.

Jensen, Johan Ludvig William Valdemar, 87–88, 377.

inequality, 4, 16, 27, 377, 382, 390, 396–397.

Jepsen, Charles Henry, 460, 525.

Jerrum, Mark Richard, 335.

Jewett, Robert Israel, 691.

Jiggs, B. H. (pen name of Baumert, Hales, Jewett, Imaginary, Golomb, Gordon, and Selfridge), 45.

Jigsaw puzzles, 139.

Jigsaw sudoku puzzles, 78, 129, 130, 160.

Job shop scheduling problems, 299, 356–357.

Jocelyn, Julian Robert John, 91, 457.

Jockusch, William Carl, 482.

Johnson, David Stifler, 11, 368, 545, 555, 670.

Join of families (ℱ ⊔ 𝒢), 17.

Join of graphs, 346.

Joint distribution, 13, 24, 380, 396–397.

Joint entropy, 24.

Jones, Alec Johnson, 423.

Jones, Kate (= Katalin Borbála Éva Ingrid Adrienne née Eyszrich), 449, 450.

JRM: Journal of Recreational Mathematics, published 1970–2014.

Jumping into the middle of a loop, 446, 475.

k-cliques, 17.

k-colorable graphs or hypergraphs, 668.

k-induction, 567.

k-wise independence, 1, 13.

k-wise ordering, 126.

Km,n (complete bipartite graphs), 107, 360, 470, 614, 618.

Kn (complete graphs), 102, 108, 120–121, 148, 151, 154, 268, 335, 337, 362, 481, 550, 626.

Kμ: One kilomem (one thousand memory accesses), 282.

Kadner, Franz, 159.

Kadon Enterprises, 449, 454, 518.

Kajitani, Yoji (), 523.

Kakuro, 155, 174, 180–182.

Kalai, Gil (), 389.

Kallenberg, Olav Herbert, 381.

Kamath, Anil Prabhakar (), 199.

Kanamoto, Nobuhiko (), 535.

Kane, Daniel Mertz, 647.

Kaplan, Craig Steven, 157.

Kaplansky, Irving, 418.

Kaporis, Alexis Constantine Flora (), 235.

Karmarkar, Narendra Krishna (), 199.

Karp, Richard Manning, 236, 384, 632.

Karpiński (= Karpinski), Marek Mieczyslaw, 588.

Karpovsky, Mark Girsh, 649.

Kasif, Simon (), 629.

Kasteleyn, Pieter Willem, 17.

Katona, Gyula (Optimális Halmaz), 291.

Katz, Daniel Jason, 531.

Kautz, Henry Alexander, 263, 316.

Kautz, William Hall, 465.

Kaye, Richard William, 571.

Keller, Michael, 132, 431, 494, 505.

Keller, Robert Marion, 267.

Kelly, John Beckwith, 526.

Kelvin, Lord [= William Thomson, 1st Baron Kelvin], 82.

Kendall, David George, 381.

KenKen®, 174–176.

Kennedy, Michael David, 34.

Kenworthy, Craig, 507.

Kern, Jerome David, iv.

Kernelization, 108.

Kernels of a digraph, 668.

Kernels of a graph (maximal independent sets), 145, 283, 318, 436, 461, 550, 552, 582, 668.

clauses for, 298, 318.

Khinchin, Alexander Yakovlevich (), inequality, 29.

Kilomem (Kμ): One thousand memory accesses, 77, 223, 282.

Kim, Jeong Han (), 238.

Kim, Scott Edward, 173, 527.

King, Benjamin Franklin, Jr., 30.

King moves, 145–146, 176, 318, 353.

King paths, 50–51, 54, 59, 136.

Hamiltonian, 535.

Kingsley, Hannah Elizabeth Seelman, 413.

Kingwise connected cells, 354, 503.

Kint-Bruynseels, Ronald Odilon Bondewijn, 510.

Kirchhoff, Gustav Robert, 87–88.

Kirousis, Lefteris Miltiades (), 235.

Kitagawa, Satoshi (), 628, 631.

Kitchiner, William, 64.

Klarner, David Anthony, 430, 483, 497, 513, 526.

Kleber, Michael Steven, 409.

Kleiman, Mark Philip, 395.

Kleinberg, Jon Michael, 392.

Kleine Büning (= Kleine-Büning), Hans Gerhard, 315, 549.

Kleitman, Daniel J (Isaiah Solomon), 640.

Knapsack problem, 669.

with a partial ordering, 342.

Knessl, Charles, 589.

Knight and bishop sudoku, 146.

Knight moves, 51, 145–146, 155, 299, 353.

Knopfmacher, Arnold, 469.

Knopp, Konrad Hermann Theodor, 87–88.

Knuth, Donald Ervin (), ii, iv, vi, viii, ix, xvii, 46, 54, 55, 63, 73, 77–79, 100, 118, 123, 185, 198, 200, 203, 235–236, 256, 258, 277, 278, 302, 309–311, 384, 389, 397, 401, 406, 411, 413, 419, 420, 424–425, 427, 429, 431–432, 440, 446, 459, 460, 463, 466, 473, 475–478, 480, 481, 484, 485, 501–503, 508, 511, 514, 523, 527, 532, 533, 538, 540, 541, 544, 548, 556, 557, 559, 561, 566, 574, 576, 577, 580, 583, 584, 591, 599–601, 604, 606, 613, 624, 628–631, 638, 639, 642, 643, 646, 650, 654, 680, 686, 714.

Knuth, John Martin (), see Truth.

Knuth, Nancy Jill Carter (), 63, 157.

Knutsen, Theodor Skjøde, see Skjøde Skjern.

Kojevnikov, Arist Alexandrovich(), 644.

Kolipaka, Kashyap Babu Rao (), 274, 345, 619.

Kolmogorov, Andrei Nikolaevich (), 9.

inequality, 9.

Komisarski, Andrzej, 371.

Konev, Boris Yurevich (), 645.

Kopparty, Swastik (), 392.

Kouřil, Michal, 189, 549.

Kowalewski, Waldemar Hermann Gerhard, 457.

Kroening, Daniel Heinrich Friedrich Emil, 567, 652.

Krom, Melven Robert, clauses, see 2SAT problems.

kSAT, 187, 233–235, 330, 332, 334, 367, 667.

Kugelpyramide puzzle, 518.

Kulikov, Alexander Sergeevich (), 644.

Kullback, Solomon, 390.

divergence (D(y||x)), 24–25, 407.

Kullmann, Oliver, 189, 289, 313, 331, 336, 579, 580, 582, 592, 624, 634, 637.

Künzell, Ekkehard, 512.

Kustes, William Adam, 508.

Kuwagaki, Akira (), 518.

Kwekkeboom, Cornelis (= Kees) Samuël, 566.

l_1 norm (|| ...||1), 393.

L(2, 1) labeling of graphs, 320.

L_7 lattice, 619.

L-bert Hall, 166.

L-cube puzzle, 509.

L pentomino, 80, see Q pentomino.

L-twist, 82, 174, 528.

La Vallée Poussin, Charles Jean Gustave Nicolas de, 388.

Labeled pyramids, 346.

Labeled traces, 346.

Lake Wobegon dice, 12.

Lalas, Efthimios George (), 235.

Lamping, John Ogden, 424.

Lamport, Leslie B., 208, 568.

Land mines, 326.

Landau, Edmund Georg Hermann, 87–88.

Landman, Bruce Michael, 549.

Langford, Charles Dudley, 140–141, 454–455.

pairs, 34–36, 55–56, 70, 105, 110–112, 118, 122, 125, 126, 150, 152, 154, 189–190, 218, 282, 305, 309, 318, 354, 550, 653.

langford(n), 190, 218–219, 223, 281, 282, 298, 305, 318, 574, 600, 653.

langford′(n), 190, 282, 298, 318, 653.

langford″(n), 282.

langford′″(n), 628.

Lapko, Olga Georgievna (), 714.

Large deviations, see Tail inequalities.

Larrabee, Tracy Lynn, 197, 321.

Larrie, Cora Mae, 21.

Larsen, Michael Jeffrey, 483.

Las Vegas algorithms, vii, 343–344.

Last block of a set partition, 101–102.

Last in, first out, 194.

Late Binding Solitaire, 298, 364.

Latin rectangle construction, 335.

Latin squares, 52, 78, 174–176, 533, 669.

Lattices, 619, 640.

of partial assignments, 349–350.

Lauria, Massimo, 240.

Laurière, Jean-Louis, 479, 550.

Lavery, Angus, 166.

Law of large numbers, 393.

Laxdal, Albert Lee, 45.

Layouts, see Floorplans, Tilings.

Lazy data structures, 214–218, 220, 249, 340, 598.

Le Berre, Daniel Claude Yves, 316.

Le Nombre Treize, see Royal Aquarium Thirteen Puzzle.

Learned clauses, 247–249, 254–255, 308, 316, 352.

sequence of, 254, 340.

Learning a Boolean function, 198–200, 299, 321.

Learning a probability distribution, 27.

Least common ancestor, 617.

Least common multiple, 23.

Leaves of a search tree, 101, 103, 126.

Leech, Jenifer (= Haselgrove, Jenifer Wheildon-Brown), 106, 486, 490.

Left-continuous function, 395.

Left division of traces, 269, 345.

Left factor of a trace, 345–346.

Left-right symmetry, 174, 428–429, 474, 520–521, 528.

Left shift, 25.

Left-to-right maxima or minima, 28, 394.

Lehmer, Derrick Henry, 54, 439, 440.

Leibler, Richard Arthur, 390.

divergence (D(y||x)), 24–25, 407.

Lemaire, Bernard François Camille, 646.

Lemma generation, see Clause-learning algorithms.

LEN field, 68–69, 89, 97, 101, 109, 110, 125–126, 132, 145, 150, 153, 155, 474–475, 480, 481.

Length of a trace, 269.

Lettmann, Theodor August, 549.

Level 0, 246, 250, 308, 340, 571, 597.

Levels of values, 246–250, 340, 597.

Levesque, Hector Joseph, 234.

Levine, Eugene, 639.

Levine, Jack, 497.

Lewis, Angelo John (= Hoffmann, Louis), 60, 459, 509, 510.

Lewis, Charles Howard, 500.

Lewis, Jerome Luther, 639.

Lewis, Meriwether, 30.

Lex-leader: The lexicographically smallest element, 295, 647.

LEXI-CUBES puzzle, 59.

Lexicographic order, 30, 35, 53, 57, 62, 108, 127, 150–152, 188, 209, 210, 214, 285, 289, 291, 293, 295–297, 299, 503, 516, 561–562, 646–647.

encoded in clauses, 285, 357, 358.

Lexicographic row/column symmetry, 290–291, 361, 365, 638.

Lexicographically smallest (or largest) solution, 152, 209–210, 295–297, 326, 341, 477, 646, 647.

Lexicographically smallest traces, 268, 345, 346, 614.

Leyton-Brown, Kevin Eric, 309, 317.

Li, Chu Min (), 315.

Li, Wei (), 333.

Lieb, Elliott Hershel, 615.

Life, Game of, 201–204, 281, 298, 321–323, 327, 351.

Lifting, 52–53.

Light speed in Life, 323.

Lindon, James Albert, 489.

Line graph of a graph, 331, 613.

Line puzzles, 460.

Linear equations, 124, 210, 595.

Linear extensions, see Topological sortings.

Linear inequalities, 172, 368.

encoding of, 284–285, 356, 357.

Linear hypergraphs, see Quad-free matrices.

Linear programming problems, 210, 479, 524–525.

Lines, abstracted, 290.

Link manipulations, 65–66, 96–97, 124.

Linked lists, 34–35, 404.

Links, dancing, 189, 305, 318, 572, 652.

Lipschitz, Rudolph Otto Sigismund, condition, 10.

Lisitsa, Alexei Petrovich (), 645.

List coloring of graphs, 319, 335.

List-decodable codes, 392.

List heads, 65–68, 89, 124, 480.

List merge sort, 479.

List merging, 595, 622.

Literal block distance, 256, 258, 342.

Literals, 186, 295.

flushing, 260.

internal representation, 212, 221, 250, 572, 573, 606, 621.

Litman, Ami (), 649.

Littlewood, John Edensor, 397.

Liu, Andrew Chiang-Fung (), 526.

Liu, Lily Li (), 391.

Livelock, 206–207.

LLINK field, 65–69, 89, 97–98, 124, 416.

Llunell, Albert Oliveras i, 631.

LNCS: Lecture Notes in Computer Science, inaugurated in 1973.

Load balancing, 54.

Loaded dice, 24.

Local equivalence, 106–108.

Local Lemma, 265–274, 317, 335, 344–349.

Local maximum, 26.

Local resampling, 266–267, 348–349.

Log-concave sequences, 14, 25, 462.

Log-convex sequences, 25.

Log encodings, 282–283, 298–299, 357.

Logemann, George Wahl, 215–216, 314, 684.

Logic puzzles, 174–183; see also Sudoku.

Longest paths and cycles, 669.

Longest simple path, 207, 567.

Lonlac, Jerry, 653.

Look-back, see Backjumping.

Lookahead, 38, 44, 54, 59.

Lookahead autarky clauses, see Black and blue principle.

Lookahead forest, 226–228, 329–331, 352.

Lookahead solvers, 222–230, 239, 281, 313, 315, 360.

combined with CDCL solvers, 313.

compared to CDCL solvers, 282–284, 302–305, 366, 654.

Loop, running time of, 21.

Loopless shadows, 368.

Loops (arcs or edges from vertices to themselves), 24.

Loops (cyclic paths), 143, 177–180, 537–538.

Loose Langford pairs, 56.

Lopsidependency graphs, 266, 267, 344, 348, 349, 549, 588.

Lord, Nicholas John, 374.

Lou, Jørgen, 515.

Lou, Xingliang David (), 416.

Lovász, Lászlό, 265, 266, 549, 555.

Loveland, Donald William, 216, 314, 684.

Lower bounds for resolution, 241–244, 337–338.

Lower semimodular lattices, 619–620.

Loyd, Samuel, 627.

Lozenges, see Diamonds.

Luby, Michael George, 264, 343.

Lucas, François Édouard Anatole, 53, 134, 399, 418, 431.

numbers, 471, 481.

Lukács, Eugene (= Jenő), 372.

Luks, Eugene Michael, 297.

Lunnon, William Frederick, 529.

m×n parallelograms, 446.

Mf and Mp, 252.

Mμ: One megamem (one million memory accesses), 46, 253, 282, 305–307.

Maaren, Hans van, 221, 230.

MacColl (= McColl), Hugh, 591.

MacMahon, Percy Alexander, 91, 110, 137–141, 155, 447, 457.

Master Theorem, 614, 615.

MacQueen, James Buford, 382.

Macro instructions, 82.

Mader, Adolf, 639.

Madigan, Conor Francis, 316.

Magen, Avner (), 54.

Magic, 557.

Magic blocks, 542, 544.

Magic masks, 372.

Magic sequences, 649.

Magic squares, 424.

Magmas, see Binary operators.

Magnetic tape, 216.

Magnification of polyforms, 159, 162.

Mahler, Kurt, 373, 409.

Makespan, 356–357.

Malik, Sharad (), 316.

Mallach, Sven, 653.

Manber, Udi (), 54.

Maneva, Elitza Nikolaeva (), 350, 620.

Mansour, Toufik (), 469.

Manthey (Siebert), Bodo, 396.

Mapping three items into two-bit codes, 363.

Mappings of {1,...,n} into {1,...,m}, 138.

march solver, 224, 580.

Marcinkiewicz, Jόsef, inequality, 29.

Marek, Victor Wiktor, 580.

Marginal costs, 118.

Marino, Raffaele, 278.

Markov (= Markoff), Andrei Andreevich (), the elder, 4, 87–88.

inequality, 4, 5, 16, 342, 383, 390, 605.

Markov, Igor Leonidovich (), 296, 645, 648.

Markström, Klas Jonas, 654.

Marlow, Thomas William, 502.

Marques da Silva (= Marques-Silva), João Paulo, 316.

Marriage theorem, 588.

Marshall, William Rex, 526.

Martin, Alexander, 628.

Martingale differences, see Fair sequences.

Martingales, 6–11, 18–20, 24, 58, 376.

with respect to a sequence, 7, 19, 382.

Masks, 398.

Mason, Perry, 30.

Masyu, 174, 178–180.

Matching, three-dimensional, see 3D matching problem.

Matching polynomial of a graph, 613.

Matchings in a graph: Sets of disjoint edges, 334, 594, 613, 668.

perfect, 102–103, 107, 120–121, 125, 148, 154, 293–294, 361, 470, 668.

Math. Comp.: Mathematics of Computation (1960– ), a publication of the American Mathematical Society since 1965; founded by the National Research Council of the National Academy of Sciences under the original title Mathematical Tables and Other Aids to Computation (1943–1959).

Mathematicians, 87–88, 132.

Mathews, Edwin Lee (41), 251.

Mathews, Harry, 437.

Matrices of 0s and 1s, 27, 66, 83–84, 98–99, 124, 126, 127, 146, 290–293, 335, 360–361, 365, 667–668; see also Grid patterns.

Matrix multiplication, 624.

Matroids, 640.

Matsui, Tomomi (), 422.

Mauro, David Whittlesey, 556.

Max-flow min-cut theorem, 22, 536.

Maximal elements of family f (f↑), 240, 246, 281, 299, 337, 341, 351, 545–546.

Maximal independent sets, see Kernels of a graph.

Maximal inequality, 8–9, 20.

Maximal planar graphs, 550.

Maximé, Oriel Dupin, 130.

Maximum-cost solutions, 158, 491.

Maximum independent sets, 271, 320, 551, 552.

Maximum number of 1s, 290–293, 319, 320, 361.

MAXSAT lower bound, 368.

“Maybe” state, 204.

Mayblox puzzle, 457.

Mayer-Eichberger, Valentin Christian Johannes Kaspar, 633.

Mazurkiewicz, Antoni Wies law, 267.

MCC problem: Multiple covering with colors, vi, 93–95, 123, 144–146, 431, 432, 442, 443, 459–463, 472, 494, 495, 498, 517, 547.

McCall’s, 64, 674.

McColl (= MacColl), Hugh, 591.

McComb, Jared Bruce, 458.

McDiarmid, Colin John Hunter, 10.

McDonald, Gary, 442, 548.

McFarren, Courtney Parsons, 507.

McGregor, William Charles, 191, 552.

graphs, 191–192, 298–299, 318–320, 552.

McGuire, Gary Mathias, 75, 76, 129.

McIlroy, Malcolm Douglas, 401, 402.

Mean, see Expected value.

Mean running time, 304.

Measure theory, 383.

Mebane, Palmer Croasdale, 538.

Median function (〈xyz〉), x, 24, 193, 320, 363, 389, 545.

Median running times, 283, 304–308, 311.

Median value of a random variable, 14, 27, 393.

Meet of families (F ⊓ G), 17.

Meeus, Jean, 490, 494, 510.

Megamem (Mμ): One million memory accesses, 253, 282, 307.

Mehlhorn, Kurt, 396.

Méjean, Henri-Michel, 590.

Mellin, Robert Hjalmar, 87–88.

transforms, 335.

MEM, an array of “cells”, 39–46, 57–58, 250, 252, 259, 308, 339–340, 603–604.

Mem (μ): One 64-bit memory access, 32, 73, 101, 218, 305.

Memo cache, 120–122, 244, 480–481, 597.

Memoization technique, 597.

Memory constraints, historic, 404.

Memoryless property, 608.

Ménage problem, 125, 481.

Menagerie, 300–301.

Mendelsohn, Nathan Saul, triples, 432.

Mengden, Nicolai Alexandrovitch von (), 29.

Mepham, Michael Andrew, 425.

Mercer, Leigh, 485.

Merge networks, 630.

Merging lists, 595, 622.

Merkle, Milan Jovan, 395.

Mertens, Stephan, 235.

Message passing, 274–279, 349–350.

METAFONT, 714.

METAPOST, 714.

Method I, 245.

Method IA, 245, 338.

Method of bounded differences, 10.

Method of Trees, 313.

Methuselah solitaire, 646.

Methuselahs in Life, 203.

mex (minimal excludant) function, 423.

Meyer auf der Heide, Friedhelm, 396.

Mézard, Marc Jean Marcel, 235, 274, 275, 279.

Michael, T. S. (born Todd Scott), 514.

Michel, Bastian, 444.

Midpoint inequalities, 630.

Mijnders, Sid, 577.

Mikusiński, Jan Stefan Geniusz, Cube, 509.

Miller, George Arthur, 514.

Miller, Jeffrey Charles Percy, 474.

Mills, Burton Everett, 314.

Minato, Shin-ichi (), 123, 484.

Minesweeper, 326–327.

Minhash algorithms, 387.

Minimal-clue puzzles, 129, 178–179, 542.

Minimal elements of a family of sets, 537, 540.

Minimal excludant (mex), 423.

Minimally dominant search trees, 105, 150.

Minimally unsatisfiable clauses, 334, 337.

Minimax solutions, 133, 402, 435, 476.

Minimum-cost exact covers, vi, 111–118, 123, 152–154, 491.

Minimum covers, 557.

Minimum cutsets, 536.

Minimum remaining values heuristic, see MRV heuristic.

Minimum spanning trees, 669.

Minimum vertex covers, 668.

Minirows of sudoku, see Trios in sudoku.

MiniSAT, 251.

Minkowski, Hermann, 87–88, 397.

inequality, 29.

Minoux, Michel, 478.

Minterms, 363, 376.

Minwise independent permutations, 23.

Mirror images, see Reflection symmetry.

Mitchell, David Geoffrey, 234.

Miters, 305, 366.

Mitsche, Dieter Wilhelm, 235.

Mittag-Leffler, Magnus Gösta (= Gustaf), 87–88.

Mitzenmacher, Michael David, 387, 388, 395.

Mixed metaphors, 260.

Mixed-radix number systems, 632.

Miyamoto, Tetsuya (), 532.

MMIX computer, iv, viii, 342.

Mobile Life paths, 202–203, 322–323.

flipflops, 322.

Möbius, August Ferdinand,
functions, 270.

series, 270, 344, 346–347, 349, 611, 613.

strip, 157.

Mod 3 addition, 298, 363.

Mod 3 parity, 363.

Mod 4 parity, 363.

Mode of a probability distribution, 26.

Model checking, 200–201, 321–325, 363–364.

Model RB, 333.

Modifications of Algorithm 7.2.2.1C and related algorithms, 126, 127, 132, 133, 138, 183, 422, 442, 445, 542.

Modified full adders, 298, 644.

Modular lattices, 619.

Moivre, Abraham de, 19, 382.

Molinari, Rory Benedict, 536.

Moments of a probability distribution, 4, 29.

Moments of a random variable, 4, 29.

Mondrian, Piet (= Mondriaan, Pieter Cornelis), 170.

mone (−1), 606.

Moniamonds, 161.

Monien, Burkhard, 579.

Monkey wrench principle, 297, 365.

Monocubes, 82.

Monominoes, 79, 84, 157, 460, 494.

Monotone functions, 347.

Boolean, 5, 321, 380, 414, 427, 533, 645.

computing CNF from DNF, 668.

Monotone Monte Carlo method, 385.

Monotonic clauses, 189, 317–318, 341.

Monotonic paths, 292, 640.

Montanari, Andrea, 279.

Monte Carlo algorithms, vii, 261–267, 342–344.

Monte Carlo estimates, 46–51, 54, 58–59, 113, 133, 445, 447, 450.

Montmort, Pierre Rémond de, 380.

Monus operation (x−⋅y=max{0,x−y}), x, 21–22, 276, 395, 463, 553, 611, 632.

Moore, Edward Forrest, 566.

Moraleda Oliván, Jorge Alfonso, 371.

Morehead, Albert Hodges, 646.

Morel, Henri, 590.

Morgan, Christopher Thomas, 514.

Morgan, John William Miller, 507, 508.

Morgenstern, Detlef, 200.

Morris, Robert, 440.

Morse, Harold Calvin Marston, constant, 373.

Moser, Leo, 418, 468.

Moser, Robin Alexander, 266, 618.

Moskewicz, Matthew Walter, 316.

Mossel, Elchanan (), 350.

Motley dissections, 170–173.

Mott-Smith, Geoffrey Arthur, 646.

Motwani, Rajeev (), 384.

Move codes, 213–215, 218, 328, 329, 339, 572, 574.

Moves, 39.

Movies, 166.

MPR: Mathematical Preliminaries Redux, ix, 1–29.

MRV heuristic (minimum remaining values), 54, 69, 75, 82, 90, 97, 99, 104–105, 118, 125–127, 145, 149, 150, 154, 155, 419, 422, 437, 470, 481–484.

Mueller, Rolf Karl, 244, 314.

Muir, Thomas, 418.

Müller, Mike, 560.

Mulmuley, Ketan Dattatraya (), 387.

Multicommodity flows, 354.

Multigraphs, 506, 595.

Multilinear function, 270.

Multimatch® puzzles, 449, 450, 454.

Multinomial theorem, 397.

Multipartitions into distinct multisets, 146.

Multiple-precision constants, 102, 373, 656–657.

Multiplication of binary numbers, 192–193, 196–198, 298, 320, 357.

Multiplication of traces, 269, 345.

Multiplication tables of a binary operator, 132, 424.

Multiplicatively fair sequences, 19.

Multiplicities of items, see MCC problem.

Multisets, 28, 129, 131, 146, 187, 578, 588, 614.

Multivalued graph colorings, 190, 283, 551–552.

Multivariate Bernoulli distribution, 14, 18, 20.

Multivariate total positivity, see FKG inequality.

Munro, James Ian, 409.

Murata, Hiroshi (), 523.

Murray, Rick, 507.

Music, 63, 135.

Mutilated chessboard, 294, 298, 361–362, 650.

Mutual exclusion protocols, 204–208, 299, 323–325.

Mutual information, 24.

Mutzbauer, Otto Adolf, 639.

Mux operation (u? v: w), 265, 286, 336, 357, 583.

Mycielski, Jan, 444.

graphs, 137, 363.

Mysterians, 654.

Mystery text, 62.

N (the number of items), 69, 71, 417–419.

N_1 (the number of primary items), 71, 417.

n-cubes, 82, 263, 320, 332, 368, 465.

n.f.: Not falsified, 635–636.

n-letter words of English, viii, 36.

n-ominoes, 79, 129–130, 414.

N pentomino, 80, 486, see S pentomino.

n queen bees, 55, 424.

n-queens problem (dominating queens), 93–94, 144, 668.

n queens problem (independent queens), 31–34, 46–48, 53–55, 70–73, 105, 110, 112–113, 118, 122, 123, 126–128, 145, 150, 152, 209, 299, 355, 424, 646, 668.

n-tone rows, 135.

n-tuples, 55.

Nacin, David Rodriguez, 466.

Nagata, Masaaki (), 123, 484.

Nahil, Julie Ann Baker, ix.

Naked singles and pairs, 75–77, 128, 425, 474, 529.

Names of hexiamonds, 161, 499.

Names of pentominoes, 80, 156, 176.

NAND operation, 244.

NanoBingo, 12–13.

Naphthalene, 162.

Napier, John, Laird of Merchiston, 193, 357.

Natsuhara, Masanori (), 410.

Nauck, Franz Christian, 53.

Nawrotzki, Kurt, 387.

Near truth, 221–223.

Nearest common ancestors, 617, 669.

Necessary assignments, 229, 330.

Negated auxiliary variables, 289.

Negative binomial distribution, cumulative, 14.

Negative k-clauses, 341.

Negative literals, 186, 318, 337.

Negatively correlated random variables, 18, 373.

Nelson, Harry Lewis, 139.

Nemhauser, George Lann, 123.

Neo Diabolical Cube, 509.

Nested motley dissections, 173, 521.

Nested parentheses, 55, 496.

Nesting phase of lookahead, 224, 226–227, 329–331.

Netto, Otto Erwin Johannes Eugen, 87–88.

Neumann, Peter, 374.

Neville-Neil, George Vernon, III (= Vicious, Kode), xvii.

New England, 115, 116, 153, 478.

New York, 116, 153.

Newbie variables, 225.

Newton, Isaac, 374.

method for rootfinding, 580–581.

NEXT(a) (the next arc with the same initial vertex as a), 62, 414.

Niborski, Rodolfo, 554.

Niemann, John, 511.

Niemelä, Ilkka Niilo Fredrik, 289.

Nieuwenhuis, Robert Lukas Mario, 631.

Nightingale, Peter William, 629.

Niho, Yoji Goff (), 405.

Nikoli puzzles, 74, 529, 532, 535, 539, 542.

Nishino, Masaaki (), 123, 484.

Nitty Gritty puzzle, 454.

Nixon, Dennison, 511.

No-player game, 201.

No-three-in-line problem, 137, 668.

Nobel, Parth Talpur, 399.

Node, 73.

Nodes of a search tree, 31–32, 102–103, 218–219, 253, 308.

Noels, Alain, 566.

Noisy data, 365.

Nonattacking queens, 209, 299, 355, 646.

Nonaveraging sets, 298, 319.

Nonchromatic rectangles, 360–361.

Nonchronological backtracking, see Backjumping.

Noncommutative variables, 346.

Nonconstructive proofs, 241, 242, 265, 566.

Noncrossing king paths, 136.

Nondeterministic finite-state automata, 359.

Nondeterministic polynomial time, 315.

Nondeterministic processes, 204, 325, 366.

Nongreedy parameter (p), 263, 278, 302, 607–608.

Nonintersecting paths, 354.

Nonisomorphic solutions, 58.

Nonnegative coefficients, 348.

Nonnegative submartingales, 9, 382.

Nonnegatively correlated random variables, 17.

Nonominoes, 130–131, 160, 164, 165, 429.

Nonprimary items, 128, 550, see also Secondary items.

Nonsharp preference heuristic, 95, 417, 432, 472, 487, 495, 498.

Nonstraight polyominoes, 156.

NONSUB subroutine, 545.

Nonsubsets f ↗ g, 537.

Nonterminal symbols, 359.

Nontransitive dice, 12.

Normal chains, 642.

Normal deviate, 393.

Normal functions, 643.

Noshita, Kohei (), 123, 502.

Not-all-equal SAT, 549.

Notational conventions, x, 660–665.

C′ ⋄ C″ (resolvent), 238, 336.

C ⊆ C′ (subsumption), 245, 336.

F | l (F given l), 211, 280, 655.

F | L (F given L), 211, 288, 341.

F ⊢ C (F implies C), 243, 336, 337.

F ⊢1∈ , 254, 341, 359.

F ⊢1 l, 287–288, 360.

F ⊢k∈ , F ⊢k l, 359–360.

G ⊕ H (direct sum), 346, 361.

|l| (a literal’s variable), 186.

±v (v or v¯), 186.

〈xyz〈(median), x, 24, 193, 320.

x & y (bitwise AND), see AND operation.

x | y (bitwise OR), see OR operation.

x ⊕ y (bitwise XOR), see XOR operation.

x−⋅y (monus), x, 21–22, 276, 395, 463, 553, 611, 632.

x? y: z (if-then-else), 265, 286, 336, 357, 583.

w(α), 241.

w(α ⊢ ∈), 241.

||α⊢C||, 241.

Notenboom, Thijs, 501.

Novikov, Yakov Andreevich (), 254, 316.

Nowakowski, Richard Joseph, 291, 639.

NP-hard and NP-complete problems, 11, 60, 127, 134, 160, 177, 185, 187, 211, 271, 314–315, 318, 326, 335, 365–367, 422, 524, 545, 571, 632, see also CoNP-complete problems.

NRC Sudoku, see Hypersudoku.

NT (near truth), 221–223, 575.

Nuij, Wilhelmus (= Wim) Antonius Adrianus, 525.

Null clause (∈), 187, 211, 549, 655.

Null list, representation of, 217, 574.

Null partial assignment, 350.

Null set (ø), 549.

Null string (∈), 268–269, 663.

Nullary clause (∈), 187, 211, 549, 655.

Nullstellensatz, combinatorial, 23.

Number Place puzzles, 74.

Number theory, 194, 198, 321, 556.

O pentomino, 80, 176, 489–490.

O’Beirne, Thomas Hay, 158, 163, 450, 500, 503, 504.

Occurrence threshold of a graph, 344.

Octabytes, 535.

Octagons, 141.

Octahedra, 139, 162, 518.

Octants, 461.

Octominoes, 166.

Odd coordinates, 455.

Odd/even coordinates, see Even/odd coordinate systems.

Odd-even merge network, 630.

Odd-even transposition sort, 627.

Odd permutations, 582.

Odier, Marc, 454.

Odlyzko, Andrew Michael, 395.

OEIS®: The On-Line Encyclopedia of Integer Sequences® (oeis.org), 422, 423, 455, 469, 504, 516, 556, 647.

Oliveras i Llunell, Albert, 631.

Ollerton, Richard Laurance, 395.

Olson, EvaMarie, 532.

On-the-fly subsumptions, 308, 340.

One-in-three satisfiability, 367.

One-per-clause satisfiability, 367.

One-sided estimates, 16.

One-sided polyforms, 110, 158, 161, 163, 487, 499–502.

Online algorithms, 153–154.

Onnen, Hendrik, Sr., 34.

Open shop scheduling problems, 299, 356–357.

Operations research, 123.

Optimization, 54.

Optimum linear arrangment problem, 669.

Optional stopping principle, 8, 382.

Options, v, 66, 88, 123.

duplicate, 98, 145, 147, 421.

three per item, 127.

without primary items, 126.

OR operation, 193, 194, 197, 622.

bitwise (x | y), 17, 128, 227, 560, 605, 622–623.

Oranges, stacking, 168–169.

Orbits of a permutation group, 292, 641.

Order encoding, 282–285, 298, 304, 354–357, 554, 632, 645.

Order ideals, 386, 427.

Order of a dissection, 171.

Order of a permutation, 295.

Order of primary items, 72, 108, 126, 150.

Ordered options, see Pairwise ordering trick.

Ordered partitions into distinct parts, 180–181.

Ordered ZDDs, 155, 480.

Organ-pipe order, 72, 355, 483.

Organ sounds, 63.

Orgel, Leslie Eleazer, 37.

Oriented cycle detection, 624.

Oriented grids, 62.

Oriented trees, 63, 292.

Orphan patterns in Life, 323.

Orponen, Olli Pekka, 264.

Orthogonal 4×4 matrices, 169.

Orthogonal arrays, 669.

Orthogonal lists, 56.

Orwell, George (= Blair, Eric Arthur), 183.

Oscillators in Life, 203, 322–323.

OSPD4: Official SCRABBLE® Players Dictionary, 36, 56, 133, 413.

Östergård, Patric Ralf Johan, 490, 514.

Ouellet, Joséphine née Quart, 440.

Oulipo, 437.

Ourotoruses, 134, 437.

Out-degree of vertex v (d+(v)), 436, 438–439, 529.

Output states, 359.

OVAL array, 250–252, 258–260, 309, 601, 604.

Overfitting, 366.

Overflow in arithmetic, 228, 251, 604.

Overflow of memory, 40, 44.

Owen, Brendan David, 505.

Owen, Mark St. John, 518.

Oxusoff, Laurent, 579.

℘ (power set, the family of all subsets), 379, 537.

℘ (tautology, the always-true clause), 187, 242, 244, 336, 364, 579, 590–592, 622.

P = NP (?), 185.

P_0(), 30.

Pm ⊠ Pn (king-move graph), 145, 534.

Packed integers, 484.

Packing problems, 125, see Exact cover problem.

Pairwise independent random variables, 1, 13.

Pairwise ordering trick, 72, 126, 174, 420, 430.

Paley, Raymond Edward Alan Christopher, 24, 389.

Palindromes, 320, 401, 402.

Panagiotou, Konstantinos (), 585.

Papadimitriou, Christos Harilaos () 261, 604, 605.

Paradoxes, 12, 13, 61, 380.

Parallel multiplication circuits, 196–198, 321.

Parallel processes, 204, 208, 305, 312–313.

Parallel programming, 54.

Parallelepipeds, 82, see Cuboids.

Parallelograms, 140–141, 446.

Parallominoes (parallelogram polyominoes), 155, 160.

Parameters, tuning of, 264–265, 277–278, 308–312, 366.

ParamILS, 309, 576, 650–651.

Parent in a tree, 63.

Parentheses, 55, 496.

Parisi, Giorgio Leonardo Renato, 278.

Parity argument, 400, 408.

Parity number, 373.

Parity of a binary integer, 13.

Parity of cells, 55, 85–86, 156–160, 163, 452, 464, 487, 491–492, 494.

Parity-related clauses, 337–338, 356, 362, 595–596, 654.

Parker, Ernest Tilden, 52.

Parker, George Swinnerton, Brothers, 83, 507.

Parliament, 131.

Partial assignments, 214, 216, 245, 246, 287, 349–350, 360, 574.

Partial backtracking, 572.

Partial fractions, 392.

Partial latin square construction, 335.

Partial orderings, 22, 240, 269, 299, 612, 640.

of dimension ≤ 2, 577.

Partial sums of random variables, 20, 28–29.

Participants, 225, 228, 329.

Partitions, 55, 59.

of a multiset, 146.

of a set, 101, 124, 138, 150, 454, 511.

of an integer, 160.

Partridge puzzle, 92–93, 144.

Patashnik, Oren, 680.

Patching and updating an algorithm, 90.

Patents, 52, 83, 91, 316, 431, 452, 454, 457, 490, 500, 510, 515, 518.

Path detection, 353.

Path dominoes, 143–144.

Path graphs Pn, 268, 344, 617.

Path length of a tree, 160.

Paths, simple, 50, 54, 59.

Patience, see Solitaire games.

Pattern design, 138, 156, 158, 162.

Paturi, Ramamohan (), 652.

Paul, Jerome Larson, 189.

Paull, Marvin Cohen, 332.

Pauls, Emil, 399.

PCk hierarchy, 360, 362.

Peaceable queens, 364.

Pearl, Judea (), 279.

Pearson, Karl (= Carl), 373.

Pegden, Wesley Alden, 348, 617.

Pegg, Edward Taylor, Jr., 459, 502, 506.

Peierls, Rudolf Ernst, 279.

Peirce, Charles Santiago Sanders, triangle, 147.

Pell, John, 416.

Pemantle, Robin Alexander, 423.

Pencil-and-paper method, 46–48.

Pendant vertex: A vertex of degree one, 506.

Pentacubes, 82, 166–167, 509.

Pentagons, 139.

Pentangle Puzzles, 510.

Pentiamonds, 161–162, 501.

Pentominoes, 62, 79–82, 106, 110, 118, 130, 139, 151, 152, 155–160, 167, 176, 494, 502, 511.

hypersolid, 515.

names of, 80, 156, 176.

shortest games, 490.

solid, 167.

wallpaper, 501.

Perelman, Grigori Yakovlevich (), 370.

Peres, Yuval (), 585.

Pérez Giménez, Xavier, 235.

Perfect matchings in a graph, 102–103, 107, 120–121, 125, 148, 154, 293–294, 361, 470, 668.

Perfect n-tone rows, 135.

Perfect Packing puzzle, 514.

Perfectly decomposed rectangles, 172.

Periodic sequences, 57.

Periodic words, 38, 41.

Perjés, Zoltan, 456.

Permanent of a matrix, 367, 615.

Permutation polynomial of a set, 347.

Permutation posets, 577.

Permutations, 27, 34, 55, 103, 107, 124, 135, 137, 140, 148, 170, 289, 543, 629.

of a multiset, 131.

signed, see Signed permutations.

weighted, 347.

Permuting variables and/or complementing them, see Signed permutations.

Perron, Oskar, 87–88.

Perturbed data, 28.

Pestieau, Jules, 485, 490.

Petamems: Quadrillions of memory accesses, 121, 582.

Petal Pushers, 435.

Peter-Orth, Christoph, 485.

Petersen, Julius Peter Christian, graph, 145.

Peterson, Gary Lynn, 207, 299, 324, 568.

Petrie, Karen Elizabeth Jefferson, 647.

Phase saving, 251, 259.

Phase transitions, 234–236, 333–334.

Phenalene, 162, 169.

Phenanthrene, 162, 169.

Phi (ϕ), 12, 121, 127, 330, 331, 344, 370, 451, 615, 656–657.

as source of “random” data, 47.

Phillips, Roger Neil, 459.

Philpott, Wade Edward (born Chester Wade Edwards), 139, 431, 450.

Phoenix in Life, 562, 571.

Pi (π), 14, 26.

as source of “random” data, 47–48, 50, 57, 60, 74, 78, 116–117, 128, 130, 146, 154, 160, 174–177, 182, 183, 196, 230, 292, 299, 331, 368, 394, 538, 557, 650; see also Pi function.

Pi day puzzle, 60, 439.

Pi function, 286–287, 358.

Picciotto, Henri (= Enrico), 504.

Pidato puzzle, 535.

Pieces, in trace theory, 268–271.

Pieces versus cells, 485.

Pierce, John Franklin, Jr., 123.

Pigeonhole principle, 241.

clauses for, 241–243, 289–290, 297, 337, 360, 365, 550, 629.

Pijanowski, Lech Andrzej, solitaire, see Dominosa.

Pikhurko, Oleg Bohdan (), 649.

Pile sums, 335.

Pinch, Ruth, 548.

Pincusians, 317.

Pinter, Ron Yair (), 523.

Pinwheels, 171–172, 462, 514, 521, 523, 527.

Pipatsrisawat, Thammanit (= Knot)(), 251, 626.

Pipe organ, 63.

Pitassi, Toniann, 54.

Pitch class, 135.

Pitman, James William, 375.

Pittel, Boris Gershon (), 384, 423.

Pixel images, 564; see also Grid patterns, Tomography.

Plaisted, David Alan, 286.

Planar graphs, 114, 536.

Planar polyspheres, 169.

Plane partitions, 482–483.

Planning, 316.

Playable sounds, 63.

Playing cards, 1, 8, 14, 19, 298, 364, 646.

Pods, 167–168.

Poetic license, 401.

Poetry, 183, 548, 666.

Poincaré, Jules Henri, 87–88, 370.

Pointing pairs, 425.

Points, abstracted, 290.

Poison cards, 646.

Poison list, 44–45, 58, 407.

Poisson, Siméon Denis, 393.

probability distribution, 15, 24, 379, 589.

trials, 379.

Polarities, 187, 251, 260, 571, 601.

Political districting, 131.

Pöllänen, Antti Ensio, 514.

Pόlya, György (= George), xiii, 6, 19, 391, 397, 399.

theorem, 648.

urn model, 6–7, 19–20, 382.

Polyaboloes, 163.

Polycrunches, 518.

Polycubes, 82–86, 127, 164–168, 174.

Polyforms, 156, 163, 502.

of polyforms, 158, 162.

Polyhedron, wrapping a, 139, 157, 162, 451.

Polyhexaspheres, 170.

Polyhexes, 162–163, 174, 529.

Polyiamonds, 141, 161–163, 174, 503.

Polyjubes, 515–516.

Polylines, see Polysticks.

Polynomials, 23, 149; see also Chebyshev polynomials, Reliability polynomials.

in trace theory, 269.

Polyomino sudoku, 78.

Polyominoes, 62, 79, 127, 139, 156–160, 164, 414; see also Pentominoes.

convex, 130.

Polyrhons, 516.

Polyskews, 163–164.

Polyspheres, 168–170.

Polysplatts, 170.

Polysquarerhombuses, see Polyskews.

Polysticks, 163.

Polytans, 503.

Pope, Alexander, 666.

Population, 115–116, 153.

Population in Life, 203.

Portfolio solvers, 317.

Posets, see Partial orderings.

Positive autarkies, 330.

Positive j-clauses, 341.

Positive literals, 186, 318, 330.

Positively correlated random variables, 17, 372.

Postal codes, 78, 114.

Posthoff, Christian, 639.

Postl, Helmut, 171, 173, 501, 521, 527.

Postorder, 226–227, 578.

Postprocessor, 280.

Potts, Charles Anthony, 486.

Povah, Maurice James, 493.

Power series, 377, 395–396.

Practice versus theory, 293, 424.

Preclusion clauses, 283, 355, 550.

Prefix of a string, 473.

Preorder, 226–227, 578.

Preprocessing of clauses, 279–281, 287, 350–352, 366, 633, 635, 642.

Preprocessing of options, 108–111, 113–114, 151–153, 482, 484, 531, 533, 536, 538, 543.

costs, 118.

Preselection phase of lookahead, 224–226, 331.

Presidents of the United States of America, 136.

Prestwich, Steven David, 628.

Preuβer, Thomas Bernd, 34, 398.

Primary items, 71, 88, 93, 127, 495, 545.

Primary variables, 288, 289.

Prime clauses, 358, 634, 637.

Prime implicants of a Boolean function, 5, 378, 427, 533, 645.

Prime numbers, 135, 320, 471, 595, 641.

Prime square problem, 113–114, 118, 152–153.

Prime strings, 38.

Primitive motley dissections, 173.

Primitive root of a prime, 440.

Princess, 158.

Pringsheim, Alfred Israel, 272, 348.

Prins, Christian, 631.

Priority branching trees, 54.

Prisms, 164–165.

Probabilistic method, 265.

Probability distribution function, 395.

Probability distributions, 148.

Bernoulli, 14, 18, 20.

Beta, 14.

binomial, 14, 24, 377, 392–393.

Cauchy, 26.

cumulative binomial, 14–15, 375, 406.

cumulative negative binomial, 14.

geometric, 21, 24, 392, 608, 626.

joint, 13, 24, 380, 396–397.

multivariate Bernoulli, 14, 18, 20.

Poisson, 15, 24, 379, 589.

Student’s t, 393.

uniform, 1, 13, 16, 22–24, 381, 385, 389–390.

Probability estimates, 3–5, 8–9, 16.

Probability generating functions, 15, 28, 384, 392, 393.

Probability of satisfiability, 231–238.

Probability spaces, 1–2, 27.

prod (m, n), 196–198, 298, 321.

Production rules, 359.

Profile of a search tree, 31–32, 37, 56, 59, 335, 406, 434, 462.

Progress, display of, 73, 214, 329, 339.

Progress saving, 251, see Phase saving.

Projection, 3D to 2D, 166.

Projection of a path, 368.

Projection vectors, 537, 538–540.

Projective plane, 638.

Propagation, kth order, 359–360, 637.

Propagation algorithm, 411.

Propagation completeness (UC_1), 360.

Propeller, 502, 517.

Proper ancestors, 348.

Properties: Logical propositions (relations), 30, 55.

Propp, James Gary, 385, 482, 483.

Proto truth, 221, 226, 329.

Protocol, randomized, 25–26.

Prover-Delayer game, 239–240, 336–337.

PSATO solver, 313.

Pseudo-Boolean constraints, see Threshold functions.

PT (proto truth), 221, 226, 581.

Pudlák, Pavel, 239.

Puget, Jean-François, 297.

Purdom, Paul Walton, Jr., 214, 216, 315, 335, 590.

Pure cycles, 324.

Pure literals, 213, 215, 216, 218, 228, 244, 314, 319, 330, 336, 572, 579, 591, 620, 623, 632, 633, 639.

Purging unhelpful clauses, 252, 255–259, 308, 316, 341, 342, 352, 366, 368, 599.

threshold for, 258, 309, 311.

purify(p), 90.

Purifying an item, 90, 117, 154, 446, 480, 536.

Putnam, Hilary Whitehall, 193, 216, 314, 684.

Puzzle Square JP, 538.

Puzzles, vi.

design of, vi, 87, 136, 140, 144, 146, 160, 166, 174–183, 367.

fiendishly difficult, 129, 170, 454, 527, 533, 535.

maximally difficult, 129, 179.

Puzzlium Sudoku ABC, 129.

Pyradox puzzle, 517.

Pyramids in trace theory, 271, 346.

Pyramids of polyspheres, 169–170.

Pyramystery puzzle, 517–518.

Q pentomino, 80, 173, 176, 474, 489–490.

q.s.: Asymptotically quite surely, 12, 20, 21, 25, 58, 149, 333, 337, 353, 371, 392, 396, 399, 482, 483.

QDD: A quasi-BDD, 552.

Quad-free matrices, 290–291, 297, 360–361, 638, 648.

Quadrants, 461, 495.

Quadrilles, 431.

Quaintance, Jocelyn Alys, 102.

Quantified Boolean formulas, 244, 338, 412.

Quarterturn symmetry, see 90°-rotational symmetry.

Quasi-independent parameters, 470.

Quasi-uniform exact cover problems, 127.

Quasigroups, 432.

Queen bees, 55, 424.

Queen domination problems, 93–94, 144.

Queen graphs, 126, 137, 209, 283–284, 298–299, 304, 355, 364, 365, 461.

Queen moves, 70–73, 93–94, 112–113, 125–127, 137, 145–146, 461, 532.

Queens, see n queens problem.

Quenchable graphs, 363–364, 645.

Questionnaires, 61.

Queues, 404, 407–408.

Quick, Jonathan Horatio, 18, 55–56, 365.

Quilt patterns, 562.

Quimper, Claude-Guy, 636.

Quine, Willard Van Orman, 313, 314.

Quintominal dodecahedra, 451.

Quite sure events, 12, see q.s.

R(G) (Local Lemma bounds), 266, 271–274, 344, 347–349.

R pentomino, 80, 176, 489–490.

R-twist, see L-twist.

Räcke, Harald, 396.

Radio colorings, 320.

Radix-d representation, 41, 357.

Rado, Richard, 555.

Ragaller, Franz, 79.

Raghavan, Prabhakar (), 384.

RainBones puzzle, 421.

Rainbows, 129, 130, 466.

Ramakrishnan, Kajamalai Gopalaswamy, 199.

raman graphs, 595.

Ramani, Aarthi (), 296, 645, 648.

Ramanujan Iyengar, Srinivasa (), graphs, 338; see also raman graphs.

Ramos, Antonio, 259, 440.

Ramsey, Frank Plumpton, theorem, 265.

rand , 223–224, 230, 234, 299, 331, 366.

Randall, Dana Jill, 483.

Random bits, 2, 3, 5, 9, 13–15, 58, 380, 406.

biased, 196, 605.

Random choices, 196.

Random decision variables, 309–311, 339, 650.

Random domino placement, 131.

Random exact cover problems, 127.

Random graphs, 16, 18, 265.

Random number generators, 385.

Random permutations, 15, 28, 597.

Random sampling, 46.

Random satisfiability problems, 231–238, 275, 335.

2SAT, 235–238, 333.

3SAT, 223–224, 230–235, 243–244, 264, 277–278, 331–337, 606.

kSAT, 233–235, 330, 332.

Random signs, 29.

Random solutions of XCC problems, 155.

Random trials, 452, 460; see also Monte Carlo estimates.

Random variables, 1–21, 47.

Random walks, 18, 28, 46–51, 59, 261–265, 309, 607.

coalescing, 21.

on r-cycle, 22, 386.

Random words, 333.

Randomization of the input, 126, 366, 432, 574.

Randomized algorithms, 21, 25–26, 149, 261, 304, 313, 622.

Range minimum query problem, 669.

RANGE scores, 258, 309–311, 342, 603.

RAT, see Resolution certifiable clauses.

Ratio of completion, 73.

Rational summation, 26.

Rauzy, Antoine Bertrand, 315, 579.

RC problems, 547.

Reachability in a graph, 183, 353, 473, 537, 540, 669.

Reachable subsets, 62–63.

Ready list, 216.

Real roots of polynomials, 347, 613.

Real truth, 221–223.

Reason, Henry, 459.

Reasons, 246–247, 250, 256, 341, 349, 597.

Rebooting, 206.

Reckhow, Robert Allen, 245.

Recreational mathematics, vi.

Rectangle-free grids, see Quad-free matrices.

Rectangles into rectangles, 170–173.

Rectangular grids, 175, 177–183.

Recurrence relations, 101–103, 148, 335, 361, 364, 381, 385, 394, 395, 408, 411, 418, 467, 547, 553, 579, 580, 607.

in a Boolean equation, 412.

Recursion versus iteration, 55, 398.

Recursive algorithms, 55, 67, 95–98, 101, 211, 314, 330, 356, 406, 412, 550, 597.

Recycling of clauses, 250, 308.

Redistricting, 131.

Reduced ZDDs, 155.

Reducing one polyform to another, 163, 169, 503, 506.

Reduction of a decomposition, 170–172.

Reduction of clauses, 211, 327; see also Simplification of clauses.

Redundant clauses, 621.

Redundant clues, removing, 129, 178–179, 542.

Redundant literals, 249, 339–340, 596, 598.

Redundant representations, 355.

Reed, Bruce Alan, 236.

Reflected strings, 152.

Reflected ternary code, 654.

Reflection symmetry, 42, 55, 83, 91, 167, 171, 174, 296, 322, 340, 398, 428–429, 449, 452–454, 495, 498, 520, 529.

about both diagonals, 171, 174, 546.

Refutation chains, 241, 591.

Refutation trees, 336.

Refutations, 238–244, 254, 294, 336; see also Certificates of unsatisfiability.

Registers, 33, 400–401.

Regular expressions, 358–359, 424.

Regular graphs and multigraphs, 24.

Regular resolution, 239, 336, 595.

Reid, Michael, 487, 509, 526.

Reid Dalmau, Robert John (= Bobby), 460.

Reinforcement messages, 275–277.

Reingold, Edward Martin (), 54, 525.

Rejection method, 47, 406.

Relabeling, remapping, 137, 138, 449, 452.

Relaxation of constraints, 53, 128, 399.

Reliability polynomials, 5, 15, 16, 267.

Reluctant doubling, vii, 261, 264–265, 343–344.

Reluctant Fibonacci sequence, 344.

Rémond de Montmort, Pierre, 380.

Removing symmetry, see Symmetry breaking.

Renamed Horn clauses, 360, 627.

Rényi, Alfréd, 378.

Repeated clauses, 233.

Repeated edges, 506.

Repeated items, 146.

Repeated options, 146.

Replacement principle, 280.

Representation of Boolean functions, 288, see Encoding into clauses.

Representation of sets, 535.

Representing three states with two bits, 363.

Required items, 88.

Rescaled activity scores, 251.

Resende, see Guilherme De Carvalho Resende.

Residue theorem, 26, 393.

Resizing of data structures, 574.

Resolution certifiable clauses, 625.

Resolution chains, 241–243, 336, 337, 591.

Resolution of clauses, 238–249, 254, 285, 313, 314, 328, 351, 549, 579, 588, 620.

implementation of, 351.

Resolution refutations, 238–244, 254, 294, 336; see also Certificates of unsatisfiability.

extended, 244, 255, 317, 338, 352, 579.

regular, 239, 336, 595.

treelike, 239–240, 336–337.

Resolvable clauses, 348.

Resolvent (C′ ⋄ C″), 238, 314, 336.

Restarting, 264–265, 279, 309, 316.

and flushing literals, 252, 259–261, 308, 316, 341, 342, 353, 598, 610.

Restricted growth strings, 15, 150, 363, 398, 446, 454, 468, 530, 543.

Restricted pigeonhole principle, 242.

Reusing the trail, 259.

Reverse dictionaries, 56.

Reverse of a string, 111, 135, 152.

Reverse plane partitions, 483.

Reverse unit propagation, 255.

Reversible memory technique, 44, 59.

Revolving-door Gray code, 646.

Reynaud, Antoine André Louis, 396.

Reynaud, Gérard, 590.

Rhombic dodecahedra, 516.

Rhomboids, 502.

Rhombuses, 163, 451, 500, 501.

Ricci-Tersenghi, Federico, 278.

Richards, Keith, 185.

Richards, Matthew John, 518.

Rickard, John Robert, 654.

Riekstiš, Eduards (), 444.

Riesz, Marcell (= Marcel), 87–88.

Right-continuous function, 395.

Right division of traces, 269, 345.

Right factor of a trace, 345.

Right shift, 25.

Right-to-left maxima or minima, 28, 394.

Right trominoes, see Bent trominoes.

Riis, Søren Møller, 294.

Riordan, John Francis, 418.

Ripoff, Robert Iosifovich (), 191.

Risueño Ferraro, Manuel María, 450.

Ritmeester, Peter, 130.

Rivest, Ronald Linn, 430, 497.

clauses R, 188, 213, 217, 239, 254, 318, 328, 334, 366.

clauses R′, 188, 212, 261, 264, 342, 350, 360, 364, 549.

Rivin, Igor (), 399, 424.

RLINK field, 65–69, 82, 89, 90, 97, 124, 416, 434, 480.

Roberts, Fred Stephen, 320.

Robertson, Aaron Jon, 549.

Robertson, Edward Lowell, III, 409.

Robinson, Gilbert de Beauregard, 639.

Robinson, John Alan, 243, 280, 591.

Rodríguez Carbonell, Enric, 631.

Rogers, Douglas George, 497.

Rogers, Leonard James, 29.

Rogers, Samuel, 548.

Rohl, Jeffrey Soden, 123.

Rokicki, Tomas Gerhard, ix, 564.

Roofs, 169.

Rooij, Iris van, 571.

Rook moves, 145–146, 476, 570.

Rookwise connected cells, 136, 354.

Rooms and bounds, 171–172, 394.

Root node, 47.

Rosenbluth, Arianna Wright, 54.

Rosenbluth, Marshall Nicholas, 54.

Rosenthal, Haskell Paul, inequality, 29.

Rosettes, 162, 500.

Ross, Kenneth Andrew, 646.

Ross, Sheldon Mark, 5, 377.

Rotated grid, 160.

Rotating Century Puzzle, see Fool’s Disk.

Rotational symmetry, see 60°-rotational symmetry, 90°-rotational symmetry, 120°-rotational symmetry, 180°-rotational symmetry.

Rote, Günter (= Rothe, Günther Alfred Heinrich), 652.

Rotors in Life, 322.

Rounding errors, 476.

Rounds of preprocessing, 152.

Roussel, Olivier Michel Joseph, 316, 636.

Roussel, Yves, 454.

Routing, disjoint, 354.

Row and column sums, 22, 335, 515.

Rows as “options”, 66, 123.

Rowwise ordering, 482.

Roy, Amitabha (), 297.

Royal Aquarium Thirteen Puzzle, 60.

Royalties, use of, 63.

Royle, Gordon Fortune, 75.

RT (real truth), 221–223, 227.

Ruiter, Johan de, 60, 183, 425, 544.

Ruler doubling, 344.

Ruler function (ρn), x, 126, 344, 532, 609.

Ruler of Fibonaccis, 610.

Runge, Carl David Tolmé, 87–88.

Running times, 73, 98, 101, 273–274.

comparison of, 218–219, 223, 253, 281–284, 289–291, 294, 296, 302–312, 366, 368, 582, 601, 628, 645, 654.

estimates of, 46–49, 54, 58–59, 113, 133, 445, 447, 450.

mean versus median, 304.

worst case, 28, 328, 330, 338.

Runs of 1s, 210, 327, 359.

Runs of a permutation, 381.

Russell, Ed (“Red Ed”), 426, 444.

S_1(y_1,...,yp), 189–190, 584.

Sk(m, n), 234–238.

Sk,n, 233–235, 332, 333.

S≥m (a symmetric threshold function), 16.

S≤r(x_1,...,xn) and S≥r(x_1,...,xn), 192, see Cardinality constraints.

s-chains, 236–237, 333.

S pentomino, 80, 176, 489–490.

s-snares, 237, 333.

Sachs, Horst, 444.

Saddle point method, 148, 590.

Safe Combination Puzzle, see Fool’s Disk.

Sahni, Sartaj Kumar (), 632.

Saïs, Lakhdar (), 600, 653.

Sakallah, Karem Ahmad (), 296, 316, 633, 645, 648.

Salhi, Yakoub (), 653.

Salvagnin, Domenico, 422.

Sample variance, 50.

Sampling with and without replacement, 26–27, 233–234, 316, 424, 473, 590.

Samson, Edward Walter, 244, 314.

Samuels, Stephen Mitchell, 15, 375.

Sands, George William (= Bill), 525.

SAT: The satisfiability problem, vi–vii, 187.

SAT solvers, viii, 150, 185, 314–317, 405–406, 427, 466, 488, 495, 513, 538.

SATexamples.tgz, viii, 302.

Satisfiability-preserving transformations, 291–297.

Satisfiability problem, 185–368, 667.

history, 216, 243–244, 289, 313–317.

Horn clauses, 667.

thresholds for, 234–238, 275, 332–333, 585.

Satisfiable formulas, 185.

variability in performance, 219, 304–305, 312, 651.

Satisfying assignments, 185, 214, 327–328, 350, 578, 583.

Saturating addition and subtraction, x, 21–22.

Saturating ternary addition, 403.

SATzilla solver, 316–317.

Sauer, Norbert Werner, 394.

Savage, Richard Preston, Jr., 370.

Save the sheep, 175.

Say Red, 382.

Schaefer, Thomas Jerome, 652.

Scheideler, Christian, 396.

Schensted, Craige Eugene (= Ea Ea), 639.

Scherer, Karl, 520.

Scherphuis, Berend Jan Jakob (= Jaap), 458.

Schlipf, John Stewart, 637.

Schmitt, John Roger, 649.

Schneider, Wolfgang, 517.

Schoenberg, Arnold Franz Walter, 135.

Scholtz, Robert Arno, 404.

Schöning, Uwe, 262.

Schossow, Frederick Alvin, 52.

Schrag, Robert Carl, 316.

Schröder, Friedrich Wilhelm Karl Ernst, 432.

Schroeppel, Richard Crabtree, 373, 561.

Schubert, Dirk Wolfram, 422.

Schulte-Geers, Ernst Franz Fred, ix, 13, 375, 383.

Schumacher, Heinrich Christian, 53, 398.

Schwartz, Benjamin Lover, 164, 508.

Schwartz, Eleanor Louise, 502.

Schwarz, Karl Hermann Amandus, inequality, 390.

Schwarzkopf, Bernd, 646.

Scott, Alexander David, 588, 615, 616.

Scott, Allan Edward Jolicoeur, 571.

Scott, Dana Stewart, 485.

Scott, Sidney Harbron, 555.

Scoville, Richard Arthur, 346.

SCRABBLE® game, 152.

Scrutchin, Thomas, 500.

Search rearrangement, see Dynamic ordering.

Search trees, 31, 32, 35, 37–39, 46–48, 52, 54, 73, 98–100, 104–107, 126, 147, 212–213, 216–218, 239, 308, 336, 399, 406, 407, 411, 434.

direct sum of (T ⊕ T′), 105, 149–150.

estimating the size, 48–49, 58–59.

expected size, 335–336.

optimum, 108, 328.

Seats at a circular table, 125, 154, 471.

Second death, 419, 490.

Second moment principle, 4–5, 16, 24, 238, 377, 585, 586.

Secondary items, 70–71, 77, 81, 88, 93, 94, 107, 125, 126, 150, 151, 416–417, 428, 467, 481, 485, 486, 489, 494, 495, 497, 508, 545.

list of active, 417, 419.

Sedgewick, Robert, 418, 497.

Seed of a hitori puzzle, 182, 547.

Seedless hitori puzzles, 183.

Seitz, Simo Sakari, 264.

Self-avoiding walks, 50, 54, 59, 60.

Self-dual futoshiki patterns, 530.

Self-dual packing, 514.

Self-equivalent sudoku solutions, 111.

Self-reference, 60, 61, 706.

Self-subsumption, 280, 351, 352, 621.

Self-supporting Soma structures, 164.

Self-synchronizing block codes, 37.

Selfridge, John Lewis, 691.

Selman, Bart, 234, 263, 316.

Semi-queens, 399.

Semicrosses, 514.

Semidistance, 422, 506.

Semifactorial (n!!), 29.

Semimodular lattices, 619–620.

Semiperimeter, 173.

Semisymmetric quasigroups, 432.

Sentinel values, 623.

Sequential allocation, 58.

Sequential consistency, 208.

Sequential lists, 39–43, 220–221, 328, 535.

Sequents, 243.

Serial correlation coefficients, 327.

Set covers, 93, 153, 443.

Set partitioning, v, see Exact cover problem.

Set partitions, 55, 101, 124, 138, 150, 374, 376, 454, 468, 511, 584.

Set splitting, see Not-all-equal SAT.

Sets, represented as integers, 379.

Sex, 501.

SGB, see Stanford GraphBase.

Shadows of paths, 368.

Shakespeare (= Shakspere), William, 183, 673.

Shallit, Jeffrey Outlaw, 423.

Shapiro, Louis Welles, 497.

Shared resource, 25–26.

Sharp preference heuristic, 106, 125, 440, 441.

Sharp thresholds, 235–236, 333.

Sharp turns, 180.

Shattered rows, 27.

Shearer, James Bergheim, 266, 271, 344.

Sheep, 175.

Sheeran, Mary, 567.

Shephard, Geoffrey Colin, 485, 488.

Shidoku, 129.

Shifted sequences, 25, 391.

Shindo, Yoshiya (), 509.

Shinozaki, Takahiro (), 388.

Shlyakhter, Ilya Alexander (), 648.

Shmoys, David Bernard, 631.

Shor, Peter Williston, 482.

Short floating point number, 472.

Shortest paths in a graph, 626, 668.

dynamic, 59.

Shortz, William Frederic, 435.

Shuffles, 1.

SIAM: The Society for Industrial and Applied Mathematics, 568.

Sicherman, George Leprechaun, ix, 162, 488, 493, 506, 509, 515, 518, 548.

SICOMP: SIAM Journal on Computing, published by the Society for Industrial and Applied Mathematics since 1972.

Sideways sum (νx): Sum of binary digits, x, 13, 25, 127, 134, 298, 327, 363, 379, 410, 424, 481, 559, 584, 607, 609, 643.

second order (ν(2)x), 327.

Sifting, 583, 584.

Siftup in a heap, 117, 598.

Signature of a clause, 256–257, 342.

Signature of a literal, 622.

Signature of a loop, 537, 539–540.

Signature of a subproblem, 120, 154, 484.

Signature of a trie node, 400.

Signed mappings, 364–365.

Signed permutations, 188, 295, 362, 528.

involutions, 296–297, 364, 641–642.

Sillke, Torsten Jürgen Georg, 167, 511, 516, 526.

Silva, see Marques da Silva.

Silver, Stephen Andrew, 322, 564.

Simkin, Menahem Michael), 55.

Simmons, Gustavus James, 556.

Simon, Laurent Dominique, 256, 316.

Simple cycles, enumerating, 537.

Simple paths, 50, 54, 59, 177, 207–208, 324, 534.

simplex graphs, 141–143, 155, 169, 320, 483, 516.

Simplification of clauses, 249, 339, 596; see also Preprocessing of clauses.

Sims, Charles Coffin, tables, 647.

Simultaneous read/write, 325.

Simultaneous write/write, 325.

Sinclair, Alistair, 264, 343, 620.

Singh, Satnam, 567.

Single lookahead unit resolution, 289, 360.

Single-stuck-at faults, 194–198, 298, 320–321.

Singleton, Colin Raymond John, 144.

Singly linked lists, 123.

Singly symmetric queen patterns, 420–421.

Sink: A vertex with no successor, 155, 271, 473, 488, 578.

components, 292–294.

Sinz, Carsten Michael, 192, 301, 302, 319, 358, 553, 644.

Sketches, 23.

Skew Ferrers boards, 155, 160.

Skew tetromino, 82.

Skew Young tableaux, 160.

Skewed rectangle, 164.

Skip Two solitaire, 646.

Skjøde Skjern, 508.

Skor-Mor Corporation, 454.

Slack, 46, 407.

in trace theory, 272, 274, 349, 615.

SLACK field, 97–98, 145.

Slack turns, 180.

Slack variables, 71, 478.

Slim polyominoes, 160.

Slisenko (= Slissenko), Anatol Olesievitch (), 243.

Slitherlink, 174, 177–179.

Sloane, Neil James Alexander, 127, 423, 469.

Slocum, Gerald Kenneth (= Jerry), 410.

Slothouber, Gerrit Jan, 84.

SLS: Stochastic local search, 261.

SLUR algorithm, 289, 360.

Sly, Allan Murray, 235.

Small polyominoes, 160.

Smart, Nigel Paul, 450.

Smile, ix, 165, 413, 535, 571.

Smiley, Dan, 507.

Smith, Barbara Mary, 647.

Smith, Cedric Austen Bardell, 679, 683.

Smoothed analysis of algorithms, 28.

Snake dance, 322.

Snake-in-the-box cycles, 146, 161, 465.

Snake-in-the-box paths, 145.

Snakes, 236–238, 333.

Snares, 236–238, 333.

Snark graphs, 253, 331, 337, 341.

Snevily, Hunter Saint Clair, 189.

Snyder, Thomas Marshall, 78, 440.

Social distancing, 159.

Socrates, son of Sophroniscus of Alopece (), 313.

SODA: Proceedings of the ACM-SIAM Symposia on Discrete Algorithms, inaugurated in 1990.

Soduko, see Sudoku.

Soft clauses, 352.

Sohler, Christian, 396.

Sokal, Alan David, 615, 616.

Solid bent trominoes, 509.

Solid pentominoes, 167.

Solitaire games, 131, 364, 646.

Solutions, number of, 232, 583.

Soma cube, 82–83, 85–86, 164–166, 485, 508, 517.

Somap, 164.

Somenzi, Fabio, 600.

Somos, Michael, 469.

Sörensson, Niklas Kristofer, 251, 339, 567, 632.

Sorkin, Gregory Bret, 235.

Sorting, 434, 479, 484, 530.

Sorting networks, 299, 321, 567, 627, 630.

Source: A vertex with no predecessor, 155, 271, 488, 616.

Spacer nodes, 68, 151, 417, 434, 474, 475, 479.

Spaceships in Life, 323, 565.

Spanning trees, 62, 545, 645, 654, 669.

Sparse binary vectors, 25.

Sparse encoding, see Direct encoding.

Sparse matrices, 66.

Sparse-set representation, 41, 575.

Speckenmeyer, Ewald, 315, 579.

Speedy Schizophrenia, 59.

Spence, Ivor Thomas Arthur, 654.

Spencer, Joel Harold, 265, 266, 618.

Sperner, Emanuel, k-families, 640.

Spiegelthal, Edwin Simeon, 413.

Spielman, Daniel Alan, 28.

Spin of a skewed square, 505.

Spiral order, 570.

Spitzer, Frank Ludvig, 28.

Spots Puzzle, 510.

Sprague, Thomas Bond, 33, 34, 55.

Square Dissection puzzle, 170.

Square of primes, 113–114, 118, 152–153.

Square tetracubes, 84, 510.

Squares (numbers of the form n^2), 92.

Squarish triangles, 501.

Squiggly sudoku, 78.

St. Petersburg paradox, 380.

Stable extensions, 438.

Stable labeling of digraphs, 410.

Stable Life configurations, 203, 561.

Stable partial assignments, 349–350.

Stable sorting, 484.

Stacks, 44, 151, 221–223, 227, 404, 529, 576.

Stacking the pieces, 268–269.

STACS: Symposium on Theoretical Aspects of Computer Science, inaugurated in 1984.

Stadje, Gert Wolfgang, 383.

Staircase polygons, 155, 160.

Stålmarck, Gunnar Martin Natanael, 240, 316, 337, 567, 596, 602.

Stamm-Wilbrandt, Hermann, 315.

STAMP(l) field, 622.

Stamping (time stamps), 44–47, 58, 221–222, 248, 250, 329, 339, 472, 575, 600, 622–624.

Standard deviation: Square root of variance, 27, 48, 50, 58, 232, 392, 604.

Stanford Artificial Intelligence Laboratory, 450.

Stanford GraphBase, iv, viii, 36, 94, 141–142, 196, 197, 310, 578, 595.

format for digraphs and graphs, 62, 413.

Stanford InfoLab, ix.

Stanford University, 646.

Stanley, Richard Peter, 483, 639.

Stappers, Filip Jan Jos, ix, 426.

Starfish graphs, 613.

Starr, Daniel Victor, 440.

Starvation, 206–208, 299, 324, 325.

Statistical mechanics, 274.

Statistics, 50.

Stators in Life, 322.

Stead, Walter, 159, 501.

Stege, Ulrike, 571.

Stein, Clifford Seth, 631.

Stein, Sherman Kopald, 432, 514.

Steinbach, Heinz Bernd, 639.

Steiner, Jacob,
tree packing, 628.

triple systems, 290, 432, 638.

Steinhaus, Władysław Hugo Dyonizy, 509.

Stellated polyhedra, 451.

Stern, Moritz Abraham, 87–88.

Stickel, Mark Edward, 316.

Sticking values, 251, see Phase saving.

Stieltjes, Thomas Jan, 87–88.

Still Life, 203, 322, 564.

Stirling, James,
approximation, 585, 604.

cycle numbers, 58.

subset numbers, 138, 234, 333, 423, 424, 468, 584, 590.

Stirzaker, David Robert, 377.

STOC: Proceedings of the ACM Symposia on Theory of Computing, inaugurated in 1969.

Stochastic local search, 261.

Stong, Richard Andrew, 440.

Stopping rules, 7, 8, 19–20.

Stopping time, 232–234, 332.

Stork, David Geoffrey, 371.

Strahler, Arthur Newell, numbers, 336.

Straight n-ominoes, 428, 430.

Straight tetracubes, 510.

Straight trominoes, 79, 84, 161, 170, 484.

Strassen, Volker, 387.

Strengthening a clause, 280, 340, 623–624.

Stříbrná, Jitka, 588.

Strichman, Ofer (), 567.

Strict exact cover problems, 98, 99, 147.

Strictly distinct literals, 186–187, 236, 349.

Strictly reduced patterns, 170, 523.

Strings generalized to traces, 267.

Strong clues, 174–175.

Strong components: Strongly connected components, 225–226, 236–237, 292, 315, 436, 578, 579, 581, 585, 627.

Strong exact coverings, 108.

Strong exponential time hypothesis, 367.

Strong product of graphs (G ⊠ H), 145, 318.

Strong solutions, 151, 157.

Strong symmetry, 138–139.

Strongly balanced sequences, 363.

Strongly three-colorable, 157, 489, 493, 497.

Stross, Charles David George, xvii.

Struyk, Adrian, 501.

Stuck-at faults, single, 194–198, 298, 320–321.

Stuckey, Peter James, 633.

Student (= William Sealy Gosset), t-distribution, 393.

Students, 411.

Stützle, Thomas Günter, 309.

Su, Francis Edward (), 184.

Subadditive law, 11, 243.

Subcubes, 332.

Subcuboids, 173.

Subforests, 226.

Subgraph isomorphism, see Embedded graphs.

Subgroups, 528.

Subinterval constraints, 554.

Sublevel sets (It), 395.

Submartingales, 8–9, 20.

Submatrices, 290–293, 361.

Submodular set functions, 379.

Subsequence of a martingale, 18.

Subset sum problem, 631–632.

Substitution, 621.

Substrings, 57.

Subsumption of clauses, 245, 280, 308, 336, 339, 340, 350–352, 365, 633.

implementation, 351, 623.

on-the-fly, 308, 340.

Subtraction, encoding of, 284.

Subtrees, 48, 54.

Sudoku, vi, 74–79, 102, 111, 128–130, 136–137, 150, 154, 174, 367, 418, 424, 474.

setup program, 425.

Sullivan, Francis Edward, 548.

Summation, rational, 26.

Summation by parts, 232.

Summers, Jason Edward, 564.

Sun, Nike (), 235.

Super Dom puzzle, 452.

Super Heads & Tails puzzle, 452.

Superdips, 403–404.

Superdominoes, 91.

Supermartingales, 8, 385.

Superpolynomially small, 12, 385.

Supertiles, 445, 448, 488.

Support clauses, 283, 298, 355.

Supported sets, 17–18.

Surprise, 178.

Survey propagation, 235, 274–279, 349–350, 577.

SWAC computer, 34.

Swaminathan, Ramasubramanian (= Ram) Pattu (), 637.

Swapping to the front, 575, 606.

Swastika, 535.

Sweep of a matrix, 292–293, 361.

Swift, Howard Raymond, 452.

Swoop of a matrix problem, 293.

Syllogisms, 313.

Sylvester, James Joseph, 87–88, 374.

Symeater in Life, 564.

Symmetric Boolean functions, 16, 363, 571, 583, 634; see also Cardinality constraints.

S≤1, see At-most-one constraint.

S_1, 189–190, 584.

S≥1, see At-least-one constraint.

Sr, 319, 363, 620.

Symmetric threshold functions, see Cardinality constraints.

Symmetrical clauses, 188, 289–290, 340.

Symmetrical clue placement, 178.

Symmetrical solutions, 322, 367, 638.

Symmetries of Boolean functions, 362.

Symmetrization of a random variable, 29.

Symmetry breaking (removal), 36, 42, 58, 70, 72, 81, 83, 86, 106, 126, 137, 138, 140, 144, 156, 157, 167, 171, 174, 189, 289–298, 322, 360–365, 398, 402, 428, 443, 445, 447, 449, 450, 458, 460, 463, 465, 482, 485, 487, 488, 491, 494, 507, 511, 517, 519, 528, 542, 546, 551, 552, 554–556, 602, 631, 645–647, 649, 652–654.

in graph coloring, 283–284, 298, 355, 363, 551.

Symmetry from asymmetry, 203, 565.

Symmetry types, 174, 528.

Synthesis of Boolean functions, 321, 362–363, 558.

Szabό, Sándor, 514.

Szabό, Tibor András, 588.

Szegedy, Máriό, 274, 345, 619.

Szegő, Gábor, xiii.

Szeider, Stefan Hans, 588, 648.

Szemerédi, Endre, 243.

Szpankowski, Wojciech, 589.

t-ary ballot numbers, 383.

t-distribution, 393.

T-grid, 163.

t-snakes, 237, 238, 333.

Tμ: One teramem (one trillion memory accesses), 81, 294, 305, 310–311, 629, 654.

Tableaux, 160, 639.

Tables of numerical quantities, 656–657.

Bell numbers (ϖn), 101.

Bernoulli numbers (Bn), 658.

Fibonacci numbers (Fn), 658.

Gould numbers (ϖ︷n), 101, 147.

Harmonic numbers (Hn), 658–659.

kCNF function counts, 584.

Queen bee numbers (H(n)), 399.

Queen numbers (Qn), 33, 420.

van der Waerden numbers (W (j, k)), 189.

Taga, Akiko (), 628, 631.

Tagged vertices, 413–414.

Tail inequalities, 4, 8–11, 15, 20, 21, 27, 391, 392, 406.

Tail of a set partition, 101–102, 148, 468.

Tait, Peter Guthrie, 418.

Tajima, Hiroshi (), 284.

Tak, Peter van der, 259.

Takaki, Kazuya (), 588.

“Take account”, 221, 227, 229–230, 581, 599.

Takei, Yoshinori (), 388.

Takenaka, Sadao (), 518.

Takizawa, Kiyoshi (), 516.

Tamura, Naoyuki (), 284, 355, 628, 631, 632.

Tangrams, 503.

Tanjo, Tomoya (), 632.

Tantalizer, see Instant Insanity.

Tantau, Till, 396.

TAOCP: The Art of Computer Programming, iv, vi, vii, xvii, 28, 299, 353.

Tape records, 216.

Tardos, Éva, 392.

Tardos, Gábor, 266, 588, 618.

Tarjan, Robert Endre, 225, 226, 545, 578, 581.

Tarnished wires, 197, 557.

Tatami tilings, 157, 171–172, 299, 327, 432, 488, 497, 520.

Tau function, 331.

Taub, Mark Lance, ix.

TAUT: The tautology problem, 187, 313, 314.

Tautological clause (℘), 187, 242, 244, 336, 364, 579, 590–592, 622.

Taxes, 118, 153.

Taylor, Brook, formula, 20.

Tee tetromino, 82.

Teng, Shang-Hua (), 28.

Tensors, 326, 335, 456.

Teramem (Tμ): One trillion memory accesses, 36–37, 224, 290, 291, 294, 305, 581, 582, 650.

Ternary clauses, 187–190, 220, 302, 315, 367; see also 3SAT problem.

Ternary commafree codes, 38–39, 41, 406.

Ternary constraints, 134.

Ternary numbers, 284, 325, 363.

Ternary operations, 193, 320.

Terpai, Tamás, 389.

Territory sets, 268, 345, 347.

Tessellations, 502.

Test cases for SAT, 297–308.

capsule summaries, 298–299.

Test patterns, see Fault testing.

Tetra puzzle, 518.

Tetracubes, 82, 166.

Tetrad tiles, 91, 138.

Tetrahedra, 143, 169, 516.

Tetrahexes, 141, 162.

Tetraspheres, 169–170.

Tetrasticks, 163.

Tetriamonds, 161.

Tetris® game, 79, 268.

Tetrominoes, 79, 82, 156–159, 505.

names of, 156.

TEX, 714.

Theobald, Gavin Alexander, 554.

Theory versus practice, 293, 424.

Thoen, Adrianus Nicolaas Joseph, 129, 425, 427, 495, 501.

Thompson, Joseph Mark, 78.

Thomson, William, see Kelvin.

Three-colorable, 157, 489, 493, 497, see also Flower snarks.

Three-connected, 523.

Three dimensions projected to two, 166.

Threshold for costs, 117–118, 153.

Threshold for new color, 446.

Threshold for progress reports, 73.

Threshold functions, 284–285, 359.

Threshold of satisfiability, 234–238, 275, 332–333, 585.

Threshold parameter Θ, 310, 577, 650.

Thue, Axel, constant, 373.

Thurley, Marc, 626.

Thurston, Edwin Lajette, 454.

Tie-breakers, 258, 603.

Tiling a floor, 299, 322, 327, 563.

Tiling the plane, 138, 140–141, 162, 322, 448, 488.

Tilings, see Exact cover problem.

by dominoes, 119, 155, 294, 298, 299, 327, 361, 362.

Time stamps, see Stamping.

Timeouts, 304.

Timmermans, Eduard Alexander (= Edo), 496.

TIMP tables, 220–224, 227, 229, 328–329.

TIP(a) (final vertex of arc a), 62, 414.

Tiskin, Alexander Vladimirovich (), 514.

To-do stack, 623.

Tolstoy, Lev Nikolayevich (), 183.

Tomographically balanced matrices, 325.

Tomography, 208–210, 299, 325–327, 351, 649.

TOP, 67–69, 88–90, 120, 125, 127, 474–475, 481.

Top-down algorithms, 616.

Topological sortings, 269, 416, 612, 669.

Torbijn, Pieter Johannes, 159, 465, 494.

Torczon, Linda Marie, 41.

Toroidal tilings, 138, 454.

Torto puzzles, 136.

Toruses, x, 55, 134, 140, 318, 322, 445, 488, 564.

3D, 167–168, 513.

generalized, 488.

Tot tibi ..., 53, 183.

Total expectation, law of, 3, 15, 28.

Total variance, law of, 28.

Totally symmetric plane partitions, 482.

Totally symmetric quasigroups, 432.

Totally uncorrelated sequences, 382.

Totient function φ(n), 439–440, 663.

Touchard, Jacques, 418.

Touched clauses, 228.

Touched variables, 623.

Tovey, Craig Aaron, 334, 587.

Tower of Babel solitaire, 646.

Tower of London solitaire, 646.

Towers, 167.

Trace of a matrix: The sum of its diagonal elements, 292, 582.

Traces (generalized strings), 267–274, 345–346, 616, 618.

Trademarks, 79.

Tradeoffs, 121, 309–310.

Trading tails, 390, 396, 469, 590.

Trail (a basic data structure for Algorithm 7.2.2.2C), 246–249, 252, 256, 308, 350, 600, 602.

reusing, 259.

Training sets, 199–200, 299, 309–311, 317, 321, 366, 650.

Transcendental numbers, 373.

Transitions between states, 200–208, 359, 566, 582.

Transitive law, 240, 592.

Transposition symmetry, 113, 137, 474, 530, 531.

Traub, Joseph Frederick, 406.

Traveling Salesrep Problem (TSP), 669.

Tree-based lookahead, see Lookahead forest.

Tree function, 594.

Tree insertion, 522.

Tree-ordered graphs, 347–348.

Treelike resolution, 239–240, 336–337.

Treengeling solver, 305.

Trees as parallominoes, 160.

Trémaux, Charles Pierre, 53.

Triacontahedron, 451.

Triagonal neighbors, 518.

Triamonds, 161.

Triangle-free graphs, 137, 351.

Triangles (3-cliques), 16, 351, 602, 628.

Triangular grids, 141–142, 163, 180, 320.

coordinates for, 138–139, 163.

Triangular masyu, 180.

Triaxial symmetry, 528.

Tribonacci numbers, 580.

Trick shapes, 164.

Tricubes, 82, 166.

Tries, 36–37, 56, 400, 476.

compressed, 401–402.

Trifolia® puzzle, 449.

Triggers, 230, 310.

Trihexaspheres, 518.

Trihexes, 169.

Trioker puzzles, 454.

Triominoes, 139.

Trios in sudoku, 129.

Triplication, 159.

Triply linked trees, 424, 545.

Tripods, 167–168.

Trispheres, 169–170.

Trivalent graphs, 331, 338, 595.

Trivial clauses, 308–311, 340, 600, 603.

Trivially satisfiable clauses, 187.

Trominoes, 79, 158, 169, 170, 493.

Truemper, Klaus, 637.

Truncated octahedron, 170.

Truncation errors, 113, 476.

Truszczyński, Mirosław (= Mirek) Janusz, 580.

Truth, degrees of, 221–223, 226–227, 229–230, 580.

Truth tables, 313–314, 363, 558, 584, 641.

Trybuła, Stanisław Czesław, 371.

Tseytin, Gregory Samuelovich (), 193, 243–244, 255, 317, 336, 338, 352, 362, 579, 595, 654.

encodings, 193, 201, 285–286, 320, 357, 559.

encodings, half of, 556, 632.

Tsimelzon, Mark Boris, 318.

Tugemann, Bastian, 76.

Tuning of parameters, 308–312, 317, 366.

Tuples, 55.

Turán, Pál (= Paul), 554.

Turton, William Harry, 364.

Tweaking, 96–97.

Twelve-tone rows, 135.

Twenty Questions, 61.

Twice Dice puzzle, 510.

Twin tree structure, 172.

Twist tetracubes, 82.

Two-factor, induced, 178.

Two-layer pieces, 160–161.

Two-letter block codes, 57.

Two-level circuit minimization, 621.

Two stacks, 535.

Tyburec, Marek, 446.

UCk hierarchy, 360, 637.

UCLA: The University of California at Los Angeles, 34.

UIP: Unique implication point, 316, 597.

ULINK field, 67–69, 88–90, 95–97, 109–110, 120, 433–434, 474–475, 481.

Unary clauses, see Unit clauses.

Unary constraints, 134.

Unary representation (= order encoding), 282–285, 298, 304, 354–357, 554, 632, 645.

Uncommitting an item in an option, 90, 120, 446.

Uncorrelated sequences, 382.

Uncovering an item, 69, 90, 117, 120.

Undeletion, 65–66, 124.

Undirected graphs versus directed graphs, 63, 667.

UNDO stack, 44.

Undoing, 32–33, 35, 43–44, 58, 65, 212–215, 221–223, 279–280, 327–329, 572, 576, 581–582.

Unhiding an option, 69, 90, 117, 120, 154, 481.

Uniform deviate: A random real number that is uniformly distributed between 0 and 1.

Uniform distribution, 1, 13, 16, 22–24, 28, 343, 381, 385, 389–390.

Uniform exact cover problems, 118, 127.

Uniform probing, 387.

Uniform sampling error, 27.

Uniformly random numbers, 406.

Union-find algorithm, 539.

Union inequality, 14.

Unique implication points, 316, 597.

Unique solutions, 59, 75, 85, 128, 130–132, 140, 144, 146, 157, 160, 164, 173–183, 413, 490–491, 502, 513, 514, 517, 519, 522, 535, 640.

and NP, 127.

Uniquely satisfiable clauses, 232, 583.

Unit clauses (= unary clauses), 187, 190, 193, 197, 205, 207, 214, 215, 217, 219, 220, 250, 254, 314, 328, 335, 341, 405, 556, 569, 574, 602, 653.

Unit conditioning, 211, 280, 350, 623, 625.

Unit propagation (⊢1), 215–218, 220, 246, 249, 252, 254–255, 277, 281–283, 287–288, 316–317, 339, 341, 349, 355, 358, 600, 633, 636, 640.

generalized to ⊢k, 359.

United States Jigsaw Sudoku, 78.

United States of America graph, 114–116, 118, 153, 320.

Universal cycles, 437, 670.

Universality of Life, 201.

University of California, 34.

University of Dresden, 34.

University of Illinois, 34.

University of Tennessee, 34.

UNIX-like convention, 484.

Unlabeled set partitions, 138.

Unnecessary branches, 239, 591.

Unordered sequential lists, 39.

Unordered sets, 404.

Unpurifying an item, 90, 117, 446, 480, 536.

Unsatisfiable core, 549.

Unsatisfiable formulas, 185.

implications of, 288, 359–360.

Unsolvable problems, 314.

Unsymmetrical queen patterns, 126.

Untweaking, 96–98.

Updates, 101–103, 147–149, 434.

Upfal, Eliezer (= Eli; ), 384, 395.

Uppercase letters, 59.

Uramasyu blog, 540.

Uri, Dario, 451.

Urns and balls, 6–7, 18–20, 382, 585.

Urquhart, Alasdair Ian Fenton, 595.

Usiskin, Zalman Philip, 371.

Utility fields in SGB format, 413.

V pentomino, 80, 81, 488.

v-reachable subsets, 62–63.

VAL array, in Algorithm 7.2.2.2C, 250–252, 257–260, 597–600, 602, 604.

in Algorithm 7.2.2.2L, 221–223, 227, 580.

Valid gradings, 411.

Valid partial assignments, 349–350.

Valid puzzles, 174–175, 180.

Vallée Poussin, Charles Jean Gustave Nicolas de La, 388.

Van de Graaff, Robert Jemison, 562.

van de Wetering, Arie [= Aad], 129, 425, 427, 493, 494, 495, 501.

van der Tak, Peter, 259.

van der Waerden, Bartel Leendert, 188.

numbers, 189, see W (k_0,...,kb−1).

van Deventer, Mattijs Oskar, 654.

Van Gelder, Allen, 255, 597, 601, 627.

van Hertog, Martien Ilse, 523.

van Maaren, Hans, 221, 230.

van Rooij, Iris, 571.

van Zwieten, Joris Edward, 221.

Vandenberghe, Lieven Lodewijk André, 377.

Vandermonde, Alexandre Théophile, matrix, 372.

Vapnik, Vladimir Naumovich (), 27.

Vapnik-Chervonenkis dimension, 27.

VAR array, 222, 366, 575.

Vardi, Ilan, 399.

Variability in performance on satisfiable problems, 219, 304–305, 312, 651.

on unsatisfiable problems, 253, 305, 312, 651.

Variable elimination, 280–281, 285, 286, 313, 338–339, 350–352, 357, 358, 620–621, 623–624, 634, 636.

Variable interaction graphs, 300–302, 366.

Variables, 186.

introducing new, 187, 190, 192, 193, 197, 244; see Auxiliary variables, Extended resolution.

Variance of a random variable, 2, 4, 9, 14, 21–22, 27, 50, 58, 60, 148, 233, 342, 348, 373, 390, 392, 604, 607.

law of total variance, 28.

Vassilevska Williams, Virginia Panayotova (), 351.

Vaughan, Theresa Elizabeth Phillips, 346.

Velthuis, Frans Jozef, 714.

Venn, John, diagram, 408.

Verbeek, Cornelis Coenraadt, 465.

Verification, 200, 341; see also Certificates of unsatisfiability.

Vertex-colored tetrahedra, 143.

Vertex cover problem, 416, 545, 668.

Vertex-disjoint paths, 155, 483.

Vertex matching, 140–141.

Vertices, 125.

Viajando puzzle, 440.

Vicious, Kode (= Neville-Neil III, George Vernon), xvii.

Vidigal Leitão, Ricardo Bittencourt, 438, 439, 443.

Viennot, Gérard Michel François Xavier, 267, 268, 271, 346, 497, 613.

Vier Farben Block puzzle, 513.

Vince, Andrew Joseph, 502.

Vinci, Leonardo di ser Piero da, 191.

Virtual unswapping, 575.

Visible nodes, 484.

Visiting an object, 30, 32, 33, 35, 46, 69, 90, 97, 414, 573.

Visualizations, 300–302.

Vitushinskiy, Pavel Viktorovich (), 646.

VLSI layout, 523.

Volkov, Stanislav Evgenyevich (), 390.

von Mengden, Nicolai Alexandrovitch (), 29.

Vondrák, Jan, 392.

Voronoï, Georgii Fedoseevich (), regions, 516, 518.

Voters, 131.

Vries, Sven de, 570.

VSIDS, 316.

W (k_0,...,kb−1) (van der Waerden numbers), 188–189, 311, 317.

W pentomino, 80, 485.

W-wall, 164–166.

Waerden, Bartel Leendert van der, 188.

numbers, 189, see W (k_0,...,kb−1).

waerden (j, k; n) problem, 188–189, 216, 219, 221, 223–226, 229, 247–250, 253, 255–259, 281, 296, 299, 305, 311–313, 317, 326–329, 340, 341, 350, 351, 365, 574, 600, 620.

Wagner, Stephan, 469.

Wagon, Stanley, 514.

Wagstaff, Samuel Standfield, Jr., 554.

Wainwright, Martin James, 350.

Wainwright, Robert Thomas, 92, 93, 322, 460, 561, 562.

Wald, Abraham (= Ábrahám), 382.

equation, 20.

Walker, Robert John, 30, 33, 34, 53–54.

Walks in a graph, 624.

WalkSAT algorithm, 263–265, 277–278, 302, 309, 343–344, 366, 555, 629, 645.

Walkup, David William, 391.

Wallis, John, 467.

Wallpaper, 140, 162, 488, 501.

Walsh, Toby, 636.

Wang, Fu Traing (), 504.

Wang, Hao (), 138.

Wang, Yi (), 391.

Wanless, Ian Murray, 399.

Warmup runs, 309, 603.

Warners, Johannes (= Joost) Pieter, 632.

Warp-30 puzzle, 518.

Warrington, Gregory Saunders, 649.

Washington Monument Puzzle, see Fool’s Disk.

Wassermann, Alfred, 490.

Watanabe, Masatoshi (), 714.

Watanabe, Tomomi (), 523.

Watched literals, 214–218, 249–250, 252, 316, 328, 339, 597–600.

Watilliaux, Charles Auguste, 166.

Weak clues, 174–175.

Weak solutions, 178, 182.

Weak symmetry, 139.

Weakly forcing, 358.

Websites, iv, viii, xii, 302, 387.

Weierstrass (= Weierstraβ), Karl Theodor Wilhelm, 87–88.

Weighted factorization, 512.

Weighted graphs, 63.

Weighted items, 124, 153.

Weighted permutations, 347.

Wein, Joel Martin, 631.

Weismantel, Robert, 628.

Welch, Lloyd Richard, 37.

Wells, Mark Brimhall, 54.

Welzl, Emmerich Oskar Roman (= Emo), 342.

Wendel, James Gutwillig, 396.

Wermuth, Udo Wilhelm Emil, ix.

Wetering, Arie [= Aad] van de, 129, 425, 427, 493, 494, 495, 501.

Wetzler, Nathan David, 255, 603.

Wheatley, Henry Benjamin, 666.

Wheel graphs (Wn), 555.

Whirls, 162.

White squares, 55, see Parity of cells.

Whitehouse, Francis Reginald Beaman, 452.

Whittlesey, Marshall Andrew, 556.

whp (with high probability), see a.s.

Width of a resolution chain, 241–243, 337–338.

Wieringa, Siert, 313.

Wiezorke, Bernhard Walter, 516, 518.

Wigderson, Avi (), 241–242, 337, 384, 595.

Wildcards in kakuro, 544.

Wilde, Boris de, 577.

Williams, Richard Ryan, 634.

Williams, Virginia Panayotova Vassilevska (), 351.

Wilson, David Bruce, 238, 333, 385, 585.

Windfalls, 227, 331, 366, 581.

Windmill dominoes, 160–161.

Windmill sudoku, 429.

Windsor, Aaron Andrew, 405, 406, 442.

Winkler, Peter Mann, 375, 654.

Winn, John Arthur, Jr., 639.

Winter, Ferdinand, 457.

Winthrop Andrews, William, 402.

Wires of a circuit, 194–198, 320.

Woan, Wen-jin (), 497.

Wobble function, 235, 335.

Wolff, Elias, 510.

Wonderword puzzles, 440.

Woods, Donald Roy, 61, 413.

Word cubes, 402.

Word rectangles, 36–37, 56, 91, 94–95, 152, 153, 182.

Word search puzzles, 87–88, 132, 136, 155, 433.

Word squares, 56.

double, 133, 181, 402.

history of, 402.

Word stair puzzles, 133, 155.

Wordcross puzzles, 136.

WORDS(n), the n most common five-letter words of English, 36, 56, 62, 94, 133, 183, 401, 434, 443, 463–464, 484.

Worst-case bounds, 28, 57, 328, 330, 338, 603, 608.

Wraparound, 55.

Wrapping a polyhedron, 139, 157, 162, 451.

Write buffers, 208.

Wyman, Max, 418.

Wynn, Edward James William, 587.

X pentomino, 80, 81, 156, 485–489, 494, 511, 519.

X2C problem: Exact cover with 2-sets, 102–103, 107, 151.

X3C problem, 102.

X4C problem, 102.

XC problem, v, see Exact cover problem.

XCC problem: Exact covering with colors, v–vi, 87–92, 113, 120, 123, 132–144, 174–183, 430, 443, 514, 532.

Xeon computer, 653.

XOR operation, 193, 194, 197, 286, 320, 338, 357.

bitwise (x ⊕ y), 127, 212, 214, 321, 560, 572, 584, 605.

Xray-like projections, 208.

XSAT, see Exact (one-per-clause) satisfiability.

Xu, Ke (), 333.

Xu, Lin (), 317.

Xu, Yixin (), 619.

Y pentomino, 80, 106, 111, 150, 167, 173.

Yada, Ayato (), 535.

Yano, Tatsuo (= Ryuoh, ), 532, 538.

Yao, Andrew Chi-Chih (), 406, 525.

Yao, Bo (), 523.

Yaroslavtsev, Grigory Nikolaevich (), 644.

Yasuda, Norihito (), 123, 484.

Yeh, Roger Kwan-Ching (), 556.

Yoder, Michael Franz, 397.

Yoshigahara, Nobuyuki (= Nob) (), 510.

Young, Alfred, tableaux, 160.

Yuster, Raphael (), 624.

Yuzawa, Kazuyuki (), 535.

Z (address of the last spacer), 69, 416, 418.

Z(m, n) (Zarankiewicz numbers), 290–291, 360.

Z pentomino, 80, 485–486.

Zabih, Ramin David, 424.

Zanette, Arrigo, 570.

Zapp, Hans-Christian, 482.

Zarankiewicz, Kazimierz, 290.

quad-free problem, 290–291, 297, 360.

Závodnýy, Jakub, 560.

ZDD: A zero-suppressed decision diagram, 51, 59, 119–124, 154–155, 535, 537, 545–546.

Zebra puzzle, 134–135.

Zecchina, Riccardo, 235, 274, 275, 620.

Zhang, Hantao (), 313, 316.

Zhang, Linbo (), 714.

Zhang, Lintao (), 316.

Zhao, Ying (), 316.

Zhu, Yunshan (), 316.

Zimmermann, Paul Vincent Marie, 399.

Živković, Zdravko, 141.

ZSEV (zero or set if even), 606.

Zucca, Livio, 506.

Zuckerman, David Isaac, 264, 343.

Zwick, Uri (), 624.

Zwieten, Joris Edward van, 221.

Zygmund, Antoni, 24, 29, 389.

THIS BOOK was composed on an Dell Precision 3600 with Computer Modern typefaces, using the TEX and METAFONT software as described in the author’s books Computers & Typesetting (Reading, Mass.: Addison–Wesley, 1986), Volumes A–E. The illustrations were produced with John Hobby’s METAPOST system. Some names in the index were typeset with additional fonts developed by Yannis Haralambous (Greek, Hebrew, Arabic), Olga G. Lapko (Cyrillic), Frans J. Velthuis (Devanagari), Masatoshi Watanabe (Japanese), and Linbo Zhang (Chinese).

Code Snippets

Many titles include programming code or configuration examples. To optimize the presentation of these elements, view the eBook in single-column, landscape mode and adjust the font size to the smallest setting. In addition to presenting code and configurations in the reflowable text format, we have included images of the code that mimic the presentation found in the print book; therefore, where the reflowable format may compromise the presentation of the code listing, you will see a “Click here to view code image” link. Click the link to view the print-fidelity code image. To return to the previous page viewed, click the Back button on your device or app.