Question:
I am strongly convinced of the value of using tests that verify a complete program (e.g. convergence tests), including an automated set of regression tests.  After reading some programming books, I've gotten the nagging feeling that I "ought to" write unit tests (i.e., tests that verify the correctness of a single function and do not amount to running the whole code to solve a problem) as well.  However, unit tests don't always seem to fit with scientific codes, and end up feeling artificial or like a waste of time.
Should we write unit tests for research codes?

For many years I was under the misapprehension that I didn't have enough time to write unit tests for my code. When I did write tests, they were bloated, heavy things which only encouraged me to think that I should only ever write unit tests when I knew they were needed.
Then I started to use Test Driven Development and I found it to be a complete revelation. I'm now firmly convinced that I don't have the time not to write unit-tests.
In my experience, by developing with testing in mind you end up with cleaner interfaces, more focussed classes & modules and generally more SOLID, testable code.
Every time I work with legacy code which doesn't have unit tests and have to manually test something, I keep thinking "this would be so much quicker if this code already had unit tests". Every time I have to try and add unit test functionality to code with high coupling, I keep thinking "this would be so much easier if it had been written in a de-coupled way".
Comparing and contrasting the two experimental stations that I support. One has been around for a while and has a great deal of legacy code, while the other is relatively new.
When adding functionality to the old lab, it is often a case of getting down to the lab and spending many hours working through the implications of the functionality they need and how I can add that functionality without affecting any of the other functionality. The code is simply not set up to allow off-line testing, so pretty much everything has to be developed on-line. If I did try to develop off-line then I would end up with more mock objects than would be reasonable.
In the newer lab, I can usually add functionality by developing it off-line at my desk, mocking out only those things which are immediately required, and then only spending a short time in the lab, ironing out any remaining problems not picked up off-line.
For clarity, and since @naught101 asked...
I tend to work on experimental control and data acquisition software, with some ad hoc data analysis, so the combination of TDD with revision control helps to document both changes in the underlying experiment hardware and as well as changes in data collection requirements over time.
Even in the situation of developing exploratory code however, I could see a significant benefit from having assumptions codified, along with the ability to see how those assumptions evolve over time.


Question: How do I know when to use lead, flux-core, lead-free, or any other kind of solder out there? Do you have any tips on solder gauge for specific applications?

A great question, and since a textbook could probably be written to answer it, there's probably not going to be any single answer.  I want to provide a general answer tailored to hobbyists, and hope that people more knowledgeable can come in and tie up specifics.
Summary
Solder is basically metal wire with a "low" melting point, where low for our purposes means low enough to be melted with a soldering iron.  For electronics, it is traditionally a mix of tin and lead.  Tin has a lower melting point than Lead, so more Tin means a lower melting point. Most common lead-based solder you'll find at the gadget store will be 60Sn/40Pb (for 60% tin, 40% lead).  There's some other minor variations you're likely to see, such as 63Sn/37Pb, but for general hobbyist purposes I have used 60/40 for years with no issue.
Science Content
Now, molten metal is a tricky beast, because it behaves a bit like water: Of particular interest is its surface tension.  Molten metal will ball up if it doesn't find something to "stick" to.  That's why solder masks work to keep jumpers from forming, and why you see surface-mount soldering tricks.  In general, metal likes to stick to metal, but doesn't like to stick to oils or oxidized metals.  By simply being exposed to air, our parts and boards start to oxidize, and through handling they get exposed to grime (such as oils from our skin).  The solution to this is to clean the parts and boards first.  That's where flux cores come in to solder.  Flux cores melt at a lower temperature than the solder, and coat the area to be soldered.  The flux cleans the surfaces, and if they're not too dirty the flux is sufficient to make a good strong solder joint (makes it "sticky" enough).
Flux Cores
There are two common types of flux cores: Acid and Rosin.  Acid is for plumbing, and should NOT be used in electronics (it is likely to eat your components or boards).  You do need to keep an eye out for that, but in general if it's in the electronics section of a gadget store it's good, if it's in the plumbing section of a home supply/home improvement store, it's bad.  In general, for hobbyist use, as long as you keep your parts clean and don't let them sit around too long, a flux core isn't necessary.  However, if you are looking for solder then you probably should pick up something with a rosin core.  The only reason you wouldn't use a flux core solder as a hobbyist is if you knew exactly why you didn't need the flux in the first place, but again, if you have some solder without flux you can probably use it for hobbyist purposes without issue.
Lead Free
That's pretty much all a hobbyist needs to know, but it doesn't hurt to know about lead-free solder since things are going that way.  The EU now requires pretty much all commercially-available electronics (with exceptions for the health and aerospace industries, as I recall) to use lead-free components, including solder.  This is catching on, and while you can still find lead-based solder it can lead to confusion.  The purpose of lead-free solder is exactly the same: It's an evolution in the product meant to be more environmentally friendly.  The issue is that lead (which is used to reduce melting point of the solder) is very toxic, so now different metals are used instead which aren't as effective at controlling melting point.  In general, you can use lead-free and lead-based solder interchangeably for hobbyist uses, but lead-free solder is a bit harder to work with because it doesn't flow as nicely or at as low a temperature as its lead-based equivalent.  It's nothing that will stop you from successfully soldering something, and in general lead-free and lead-based solders are pretty interchangeable to the hobbyist.
Tutorials
There are plenty of soldering videos on YouTube, just plugging in "soldering" to the search should turn up plenty.  NASA has some old instructional videos that are great, because they deal with a lot of through-hole components.  Some of these are relevant because they discuss the techniques and how the solder types relate.
In general, if you got it at the electronics hobby shop, it's good to use for hobbyist purposes.


Question:
To try to test whether an algorithm for some problem is correct, the usual starting point is to try running the algorithm by hand on a number of simple test cases -- try it on a few example problem instances, including a few simple "corner cases".  This is a great heuristic: it's a great way to quickly weed out many incorrect attempts at an algorithm, and to gain understanding about why the algorithm doesn't work.
However, when learning algorithms, some students are tempted to stop there: if their algorithm works correctly on a handful of examples, including all of the corner cases they can think to try, then they conclude that the algorithm must be correct.  There's always a student who asks: "Why do I need to prove my algorithm correct, if I can just try it on a few test cases?"
So, how do you fool the "try a bunch of test cases" heuristic?  I'm looking for some good examples to show that this heuristic is not enough.  In other words, I am looking for one or more examples of an algorithm that superficially looks like it might be correct, and that outputs the right answer on all of the small inputs that anyone is likely to come up with, but where the algorithm actually doesn't work.  Maybe the algorithm just happens to work correctly on all small inputs and only fails for large inputs, or only fails for inputs with an unusual pattern.
Specifically, I am looking for:

An algorithm.  The flaw has to be at the algorithmic level.  I am not looking for implementation bugs.  (For instance, at a bare minimum, the example should be language-agnostic, and the flaw should relate to algorithmic concerns rather than software engineering or implementation issues.)
An algorithm that someone might plausibly come up with.  The pseudocode should look at least plausibly correct (e.g., code that is obfuscated or obviously dubious is not a good example).  Bonus points if it is an algorithm that some student actually came up with when trying to solve a homework or exam problem.
An algorithm that would pass a reasonable manual test strategy with high probability.  Someone who tries a few small test cases by hand should be unlikely to discover the flaw.  For instance, "simulate QuickCheck by hand on a dozen small test cases" should be unlikely to reveal that the algorithm is incorrect.
Preferably, a deterministic algorithm.  I've seen many students think that "try some test cases by hand" is a reasonable way to check whether a deterministic algorithm is correct, but I suspect most students would not assume that trying a few test cases is a good way to verify probabilistic algorithms.  For probabilistic algorithms, there's often no way to tell whether any particular output is correct; and you can't hand-crank enough examples to do any useful statistical test on the output distribution.  So, I'd prefer to focus on deterministic algorithms, as they get more cleanly to the heart of student misconceptions.

I'd like to teach the importance of proving your algorithm correct, and I'm hoping to use a few examples like this to help motivate proofs of correctness.  I would prefer examples that are relatively simple and accessible to undergraduates; examples that require heavy machinery or a ton of mathematical/algorithmic background are less useful.  Also, I don't want algorithms that are "unnatural"; while it might be easy to construct some weird artificial algorithm to fool the heuristic, if it looks highly unnatural or has an obvious backdoor constructed just to fool this heuristic, it probably won't be convincing to students. Any good examples?

A common error I think is to use greedy algorithms, which is not always the correct approach, but might work in most test cases.
Example: Coin denominations, $d_1,\dots,d_k$ and a number $n$,
express $n$ as a sum of $d_i$:s with as few coins as possible.
A naive approach is to use the largest possible coin first,
and greedily produce such a sum.
For instance, the coins with value $6$, $5$ and $1$
will give correct answers with greedy for all numbers between $1$ and $14$
except for the number $10 = 6+1+1+1+1 = 5+5$.


Question:
So we all know that the continued fraction containing all $1$s...
$$
x = 1 + \frac{1}{1 + \frac{1}{1 + \ldots}}.
$$
yields the golden ratio $x = \phi$, which can easily be proven by rewriting it as $x = 1 + \dfrac{1}{x}$, solving the resulting quadratic equation and assuming that a continued fraction that only contains additions will give a positive number.
Now, a friend asked me what would happen if we replaced all additions with subtractions:
$$
x = 1 - \frac{1}{1 - \frac{1}{1 - \ldots}}.
$$
I thought "oh cool, I know how to solve this...":
\begin{align}
x &= 1 - \frac{1}{x} \\
x^2 - x + 1 &= 0.
\end{align}
And voila, I get...
$$ x \in \{e^{i\pi/3}, e^{-i\pi/3} \} .$$
Ummm... why does a continued fraction containing only $1$s, subtraction and division result in one of two complex (as opposed to real) numbers?
(I have a feeling this is something like the $\sum_i (-1)^i$ thing, that the infinite continued fraction isn't well-defined unless we can express it as the limit of a converging series, because the truncated fractions $1 - \frac{1}{1-1}$ etc. aren't well-defined, but I thought I'd ask for a well-founded answer. Even if this is the case, do the two complex numbers have any "meaning"?)

You're attempting to take a limit.
$$x_{n+1} = 1-\frac{1}{x_n}$$
This recurrence actually never converges, from any real starting point.
Indeed, $$x_2 = 1-\frac{1}{x_1}; \\ x_3 = 1-\frac{1}{1-1/x_1} = 1-\frac{x_1}{x_1-1} = \frac{1}{1-x_1}; \\ x_4 = x_1$$
So the sequence is periodic with period 3.
Therefore it converges if and only if it is constant; but the only way it could be constant is, as you say, if $x_1$ is one of the two complex numbers you found.
Therefore, what you have is actually basically a proof by contradiction that the sequence doesn't converge when you consider it over the reals.
However, you have found exactly the two values for which the iteration does converge; that is their significance.
Alternatively viewed, the map $$z \mapsto 1-\frac{1}{z}$$ is a certain transformation of the complex plane, which has precisely two fixed points. You might find it an interesting exercise to work out what that map does to the complex plane, and examine in particular what it does to points on the real line.


Question:
I was wondering why dogs shouldn't eat chocolate. Can't dogs just excrete the indigestible component in their droppings?
It's common knowledge that dogs shouldn't eat chocolate. What I don't know is why chocolate would kill them, from a specifically biological perspective.

The reason is simple: Chocolate contains cocoa which contains Theobromine. The darker the chocolate is (meaning the more cocoa it contains) the more theobromine it contains. This is a bitter alkaloid which is toxic to dogs (and also cats), but can be tolerated by humans.
The reason for this is the much slower metabolization of theobromine in the animals (there are reports for poisonings of dogs, cats, birds, rabbits and even bear cubs) so that the toxic effect can happen. Depending on the size of the dog, something between 50 and 400g of milk chocolate can be fatal. As mentioned by @anongoodnurse the cocoa content in milk chocolate is the lowest and much higher the darker the chocolate gets.
The poisoning comes from the Theobromine itself, which has different mechanisms of action:
First it is an unselective antagonist of the adenosine receptor, which is a subclass of G-protein coupled receptors on the cell surface which usually bind adenosine as a ligand. This influences cellular signalling.
Then it is a competitive nonselective phosphodiesterase inhibitor, which prevents the breakdown of cyclic AMP in the cell. cAMP is an important second messenger in the cell playing an important role in the mediation of signals from the outside of the cells via receptors to a reaction of a cell to changing conditions. The levels of cAMP are tightly controlled and the half-life of the molecule is generally short. Elevated levels lead to an activation of the protein kinase A , an inhibition TNF-alpha and leukotriene synthesis and reduces inflammation and innate immunity. For references see here.
The LD50 for theobromine is very different among species (table from here), with LD50 as the lethal dose killing 50% of the individuals and TDlo the lowest published toxic dose:

The LD50 also differs between different breeds of dogs, so there are online calculators available to make an estimation, if there is a problem or not. You can find them for example here and here. The selective toxicity makes it even an interesting poison for pest control of coyotes, see reference 4 for some details.
References:

Chocolate - Veterinary Manual
Chocolate intoxication
The Poisonous Chemistry of Chocolate
Evaluation of cocoa- and coffee-derived methylxanthines as toxicants
for the control of pest coyotes.


Question:
I am new to chemistry and I find it fascinating. I am trying to learn about chemical reactions and I was wondering if there was an easy way to quickly tell if any combination of chemical substances would produce a reaction and what product(s) if any might be formed.
For example, if I pick any two random substances $\ce{A}$ and $\ce{B}$, can I determine if a reaction will occur and predict the products?
$$\ce{A + B -> \ ?}$$
More specifically, let's say I just learned that chlorine bleach (sodium hypochlorite) can be made by a reaction of sodium hydroxide and chlorine with sodium chloride as a byproduct:
$$\ce{NaOH(aq) + Cl2(g) -> NaOCl(aq) + NaCl(aq)}$$
Is there a way that I could have predicted this reaction (and any other) before I learned about it? I do not want to memorize the outcome of every combination o that I can answer questions about chemical reactions. I am hoping there is a short list of simple rules that govern all chemical reactions that I can commit to memory and then apply to any combination of substances. I might also like to be able to develop a simple computer program built around an algorithm for these reactivity rules that can sample databases of known substances and predict new reactions.

Can I predict the products of any chemical reaction?
In theory, yes!
Every substance has characteristic reactivity behavior. Likewise pairs and sets of substances have characteristic behavior. For example, the following combinations of substances only have one likely outcome each:
$$
\ce{HCl + NaOH -> NaCl + H2O} \\[2ex]
\ce{CH3CH2CH2OH->[$1.$ (COCl)2, (CH3)2SO][$2.$ Et3N] CH3CH2CHO}
$$
However, it is not a problem suited to brute force or exhaustive approaches
There are millions or perhaps billions of known or possible substances. Let's take the lower estimate of 1 million substances. There are $999\,999\,000\,000$ possible pairwise combinations. Any brute force method (in other words a database that has an answer for all possible combinations) would be large and potentially resource prohibitive. Likewise you would not want to memorize the nearly 1 trillion combinations.
If more substances are given, the combination space gets bigger. In the second example reaction above, there are four substances combined: $\ce{CH3CH2CH2OH}$, $\ce{(COCl)2}$, $\ce{(CH3)2SO}$, and $\ce{Et3N}$. Pulling four substances at random from the substance space generates a reaction space on the order of $1\times 10^{24}$ possible combinations. And that does not factor in order of addition. In the second reaction above, there is an implied order of addition:

$\ce{CH3CH2CH2OH}$
$\ce{(COCl)2}$, $\ce{(CH3)2SO}$
$\ce{Et3N}$

However, there are $4!=24$ different orders of addition for four substances, some of which might not generate the same result. Our reaction space is up to $24\times 10^{24}$, a bewildering number of combinations. And this space does not include other variables, like time, temperature, irradiation, agitation, concentration, pressure, control of environment, etc. If each reaction in the space could somehow be stored for as little as 100 kB of memory, then the whole space of combinations up to 4 substances would require $2.4 \times 10^{27}$ bytes of data, or $2.4\times 10^7$ ZB (zettabytes) or $2.4\times 10^4$ trillion terabytes. The total digital data generated by the human species was estimated recently (Nov. 2015) to be 4.4 ZB. We need $5.5\times 10^5$ times more data in the world to hold such a database. And that does not even count the program written to search it or the humans needed to populate it, the bandwidth required to access it, or the time investment of any of these steps.
In practice, it can be manageable!
Even though the reaction space is bewilderingly huge, chemistry is an orderly predictable business. Folks in the natural product total synthesis world do not resort to random combinations and alchemical mumbo jumbo. They can predict with some certainty what type of reactions do what to which substances and then act on that prediction.
When we learn chemistry, we are taught to recognize if a molecule belongs to a certain class with characteristic behavior. In the first example above, we can identify $\ce{HCl}$ as an acid and $\ce{NaOH}$ as a base, and then predict an outcome that is common to all acid-base reactions. In the second example above, we are taught to recognize $\ce{CH3CH2CH2OH}$ as a primary alcohol and the reagents given as an oxidant. The outcome is an aldehyde.
These examples are simple ones in which the molecules easily fit into one class predominantly. More complex molecules may belong to many categories. Organic chemistry calls these categories “Functional Groups”. The ability to predict synthetic outcomes then begins and ends with identifying functional groups within a compound's structure. For example, even though the following compound has a more complex structure, it contains a primary alcohol, which will be oxidized to an aldehyde using the same reagents presented above. We can also be reasonably confident that no unpleasant side reactions will occur.

If the reagents in the previous reaction had been $\ce{LiAlH4}$ followed by $\ce{H3O+}$, then more than one outcome is possible since more than one functional group in the starting compound will react. Controlling the reaction to give one of the possible outcomes is possible, but requires further careful thought.
There are rules, but they are not few in number. There are too many classes of compounds to list here. Likewise even one class, like primary alcohols (an hydroxyl group at the end of a hydrocarbon chain) has too many characteristic reactions to list here. If there are 30 classes of compounds (an underestimate) and 30 types of reactions (an underestimate), then there are 900 reaction types (an underestimate). The number of viable reaction types is more manageable than the total reaction space, but would still be difficult to commit to memory quickly. And new reaction types are being discovered all the time.
Folks who learn how to analyze combinations of compounds spend years taking courses and reading books and research articles to accumulate the knowledge and wisdom necessary. It can be done. Computer programs can be (and have been) designed to do the same analysis, but they were designed by people who learned all of the characteristic combinations. There is no shortcut.


Question: As the title says really, why do ethernet sockets need to be mag-coupled? I have a basic understanding of electronics, but mostly, I can't figure out the right search terms to google this properly.

The correct answer is because the ethernet specification requires it.
Although you didn't ask, others may wonder why this method of connection was chosen for that type of ethernet.  Keep in mind that this applies only to the point-to-point ethernet varieties, like 10base-T and 100base-T, not to the original ethernet or to ThinLan ethernet.
The problem is that ethernet can support fairly long runs such that equipment on different ends can be powered from distant branches of the power distribution network within a building or even different buildings.  This means there can be significant ground offset between ethernet nodes.  This is a problem with ground-referenced communication schemes, like RS-232.
There are several ways of dealing with ground offsets in communications lines, with the two most common being opto-isolation and transformer coupling.  Transformer coupling was the right choice for ethernet given the tradeoffs between the methods and what ethernet was trying to accomplish.  Even the earliest version of ethernet that used transformer coupling runs at 10 Mbit/s.  This means, at the very least, the overall channel has to support 10 MHz digital signals, although in practice with the encoding scheme used it actually needs twice that.  Even a 10 MHz square wave has levels lasting only 50 ns.  That is very fast for opto-couplers.  There are light transmission means that go much much faster than that, but they are not cheap or simple at each end like the ethernet pulse transformers are.
One disadvantage of transformer coupling is that DC is lost.  That's actually not that hard to deal with.  You make sure all information is carried by modulation fast enough to make it thru the transformers.  If you look at the ethernet signalling, you will see how this was considered.
There are nice advantages to transformers too, like very good common mode rejection.  A transformer only "sees" the voltage across its windings, not the common voltage both ends of the winding are driven to simultaneously.  You get a differential front end without a deliberate circuit, just basic physics.
Once transformer coupling was decided on, it was easy to specify a high isolation voltage without creating much of a burden.  Making a transformer that insulates the primary and secondary by a few 100 V pretty much happens unless you try not to.  Making it good to 1000 V isn't much harder or much more expensive.  Given that, ethernet can be used to communicate between two nodes actively driven to significantly different voltages, not just to deal with a few volts of ground offset.  For example, it is perfectly fine and within the standard to have one node riding on a power line phase with the other referenced to the neutral.


Question: During convolution on a signal, why do we need to flip the impulse response during the process?

Adapted from an answer to a different question (as mentioned in a comment) in the hope that this question will not get thrown up repeatedly by Community Wiki as one of the Top Questions....

There is no "flipping" of the impulse response by a linear
(time-invariant) system.
The output of a linear time-invariant system is
the sum of scaled and time-delayed versions of the
impulse response, not the "flipped" impulse response.

We break down the input signal $x$ into a sum of scaled
unit pulse signals. The system response to the unit pulse signal
$\cdots, ~0, ~0, ~1, ~0, ~0, \cdots$ is
the impulse response or pulse response
$$h[0], ~h[1], \cdots, ~h[n], \cdots$$
and so by the scaling property the single input value $x[0]$,
or, if you prefer
$$x[0](\cdots, ~0, ~0, ~1, ~0,~ 0, \cdots)
=  \cdots ~0, ~0, ~x[0], ~0, ~0, \cdots$$
creates a response
$$x[0]h[0], ~~x[0]h[1], \cdots, ~~x[0]h[n], \cdots$$
Similarly, the single input value $x[1]$ or
$$x[1](\cdots, ~0, ~0, ~0, ~1,~ 0, \cdots)
=  \cdots ~0, ~0, ~0, ~x[1], ~0, \cdots$$
creates a response
$$0, x[1]h[0], ~~x[1]h[1], \cdots, ~~x[1]h[n-1], x[1]h[n] \cdots$$
Notice the delay in the response to $x[1]$.  We can continue further
in this vein, but it is best to switch to a more tabular form
and show the various outputs aligned properly in time. We have
$$\begin{array}{l|l|l|l|l|l|l|l}
\text{time} \to & 0 &1 &2 & \cdots & n & n+1 & \cdots \\
\hline
x[0] & x[0]h[0] &x[0]h[1] &x[0]h[2] & \cdots &x[0]h[n] & x[0]h[n+1] & \cdots\\
\hline
x[1] & 0 & x[1]h[0] &x[1]h[1] & \cdots &x[1]h[n-1] & x[1]h[n] & \cdots\\
\hline
x[2] & 0 & 0 &x[2]h[0] & \cdots &x[2]h[n-2] & x[2]h[n-1] & \cdots\\
\hline
\vdots & \vdots & \vdots & \vdots & \ddots & \\
\hline
x[m] & 0 &0 & 0 & \cdots & x[m]h[n-m] & x[m]h[n-m+1] & \cdots \\
\hline
\vdots & \vdots & \vdots & \vdots & \ddots
\end{array}$$
The rows in the above array are precisely the scaled and
delayed versions of the impulse response that add up to
the response $y$ to input signal $x$.
But if you ask a more specific question such as

What is the output at time $n$?

then you can get the answer by summing the $n$-th column to get
$$\begin{align*}
y[n] &= x[0]h[n] + x[1]h[n-1] + x[2]h[n-2] + \cdots + x[m]h[n-m] + \cdots\\
&= \sum_{m=0}^{\infty} x[m]h[n-m],
\end{align*}$$
the beloved convolution formula that befuddles generations of students
because the impulse response seems to be "flipped over"
or running backwards in time. But, what people seem to forget
is that instead we could have written
$$\begin{align*}
y[n] &= x[n]h[0] + x[n-1]h[1] + x[n-2]h[2] + \cdots + x[0]h[n] + \cdots\\
&= \sum_{m=0}^{\infty} x[n-m]h[m],
\end{align*}$$
so that it is the input that seems "flipped over" or running
backwards in time!
In other words, it is human
beings who flip the impulse response (or the input) over when computing
the response at time $n$ using the convolution formula, but
the system itself does nothing of the sort.


Question:
It's pretty clear from a survey of the top500 that the industry is trending towards an exponential increase in processing cores.  The largest supercomputers all use MPI for communication between nodes, though there does not appear to be a clear trend for on-node parallelism, with the simplest (but not necessarily the most efficient) approach to map a single MPI process to each core, automatic parallelization from the compiler, OpenMP, pthreads, CUDA, Cilk, and OpenCL.
I am one of a group of scientists maintaining and developing a code that has potential to be used on some of the largest supercomputers in the world.  Assuming finite developer time, how do I future-proof myself so that I can take advantage of the performance of the world's most powerful machine?  What assumptions should I make about process interconnect architecture?  What paradigms are going to suffer as we enter the manycore era?  Will Partitioned Global Address Space languages be available "in production" on petascale machines?

Historical Perspective
It is really impossible to say what the new paradigms will be like in the future, for example a good historical perspective I suggest reading Ken Kennedy's Rise and Fall of HPF.  Kennedy gives an account of two emerging patterns, MPI versus a smart compiler, and details how MPI had the right amount of early adopters and flexibility to dominate.  HPF eventually fixed its problems but it was too late.
In many ways, several paradigms, such as PGAS and OpenMP, are following that same HPF trend.  The early codes have not been flexible enough to use well and left a lot of performance on the table.  But the promise of not having to write every iota of the parallel algorithm is a attractive goal.  So the pursuit of new models are always being pursued.

Clear Trends in Hardware
Now the success of MPI has often been cited as to being closely tied to how it models the hardware it runs on.  Roughly each node has a few number of processes and passing the messages to local point-to-point or through coordinated collective operations is easily done in the cluster space.  Because of this, I don't trust anyone who gives a paradigm that doesn't follow closely to new hardware trends, I actually was convinced of this opinion by the work from Vivak Sarakar.
In keeping with that here are three trends that are clearly making headway in new architectures. And let me be clear, there are now twelve different architectures being marketed in HPC.  This up from less than 5 years ago only featuring x86, so the coming days will see lots of opportunities for using hardware in different and interesting ways

Special Purpose Chips: Think large vector units like accelerators (view espoused by Bill Dally of Nvidia)
Low Power Chips: ARM based clusters (to accomodate power budgets)
Tiling of Chips: think tiling of chips with different specifications (work of Avant Argwal)

Current Models
The current model is actually 3 levels deep.  While there are many codes using two of these levels well, not many have emerged using all three.  I believe that to first get to exascale one needs to invest in determining if you code can run at all three levels.  This is probably the safest path for iterating well with the current trends.
Let me iterate on the models and how they will need to change based on the predicted new hardware views.
Distributed
The players at the distributed level largely fall into MPI and PGAS languages.  MPI is a clear winner right now, but PGAS languages such as UPC and Chapel are making headways into the space.  One good indication is the HPC Benchmark Challenge.  PGAS languages are giving very elegant implementations of the benchmarks.
The most interesting point here is that while this model currently only works at the node level, it will be an important model inside a node for Tiled architectures.  One indication is the Intel SCC chip, which fundamentally acted like a distributed system.  The SCC team created their own MPI implementation and many teams were successful at porting community libraries to this architecture.
But to be honest PGAS really has a good story for stepping in to this space.  Do you really want to program MPI internode and then have to do the same trick intranode?  One big deal with these tiled architectures is that they will have different clock speeds on the chips and major differences in bandwidth to memory so performant codes must take this into account.
On-node shared memory
Here we see MPI often being "good enough", but PThreads (and libraries deriving from PThreads such as Intel Parallel Building Blocks) and OpenMP are still used often.  The common view is that there will be a time when there are enough shared memory threads that MPI's socket model will break down for RPC or you need a lighter weight process running on the core. Already you can see the indications of IBM Bluegene systems having problems with shared memory MPI.
As Matt comments, the largest performance boost for compute intensive codes is the vectorization of the serial code.  While many people assume this is true in accelerators, it is also critical for on-node machines as well.  I believe Westmere has a 4 wide FPU, thus one can only get a quarter of the flops without vectorization.
While I don't see the current OpenMP stepping into this space well, there is a place for low-powered or tiles chips to use more light threads. OpenMP has difficulty describing how the data flow works and as more threads are used I only see this trend becoming more exaggerated, just look at examples of what one has to do to get proper prefetching with OpenMP.
Both OpenMP and PThreads at a course enough level can take advantage of the vectorization necessary to get a good percentage of peak, but doing so requires breaking down your algorithms in a way that vectorization is natural.
Co-processor
Finally the emergence of the co-processor (GPU, MIC, Cell acclerators) has taken hold.  It is becoming clear that no path to exascale will be complete without them.  At SC11, every Bell prize contestent used them very effectively to get to the low petaflops.  While CUDA and OpenCL have dominated the current market, I have hopes for OpenACC and PGAS compilers entering the space.
Now to get to exascale, one proposal is to couple the low powered chips to lots of co-processors.  This will pretty well kill off the middle layer of the current stack and use codes that manage decision problems on the main chip and shuffle off work to the co-processors.  This means that for code to work quite effectively a person must rethink the algorithms in terms of kernels (or codelets), that is branchless instruction level parallel snippets.  As far as I know, a solution to this evolution is pretty wide open.

How this affects the app developer
Now to get to your question.  If you want to protect yourself from the oncoming complexities of exascale machines, you should do a few things:

Develop your algorithms to fit at least three levels of parallel hierarchy.
Design your algorithms in terms of kernels that can be moved between the heirarchy.
Relax your need for any sequential processes, all of these effects will happen asynchronously because synchronous execution is just not possible.

If you want to be performant today, MPI + CUDA/OpenCL is good enough but UPC is getting there so its not a bad idea to take a few days and learn it.  OpenMP gets you started but leads to problems once the code needs to be refactored. PThreads requires completely rewriting your code to its style.  Which makes MPI + CUDA/OpenCL the current best model.

What is not discussed here
While all this talk of exascale is nice, something not really discussed here is getting data onto and off of the machines.  While there have been many advances in memory systems, we don't see them in commodity cluster (just too darned expensive).  Now that data intensive computing is becoming a large focus of all the super computing conferences, there is bound to be a bigger movement into the high memory bandwidth space.
This brings to the other trend that might happen (if the right funding agencies get involved). Machines are going to become more and more specialized for the type of computing required.  We already see "data-intensive" machines being funded by the NSF, but these machines are on a different track than the 2019 Exascale Grand Challenge.
This became longer than expected ask for references where you need them in the comments


Question:
Recently, I read in the journal Nature that Stephen Hawking wrote a paper claiming that black holes do not exist.  How is this possible? Please explain it to me because I didn't understand what he said.
References:

Article in Nature News: Stephen Hawking: 'There are no black holes' (Zeeya Merali, January 24, 2014).
S. Hawking, Information Preservation and Weather Forecasting for Black Holes, arXiv:1401.5761.

This is really a footnote to the accepted answer.
Light cannot escape from an event horizon. But how can you check that light can never escape? You can watch the surface for some time $T$, but all you have proved is that light can't escape in the time $T$. This is what we mean by an apparent horizon, i.e. it is a surface from which light can't escape within a time $T$.
To prove the surface really was an event horizon you would have to watch it for an infinite time. The problem is that Hawking radiation means that no event horizon can exist for an infinite time. The conclusion is that only apparent horizons can exist, though the time $T$ associated with them can be exceedingly long, e.g. many times longer than the current age of the universe.
A point worth mentioning because it's easy to overlook: when you start learning about black holes you'll start with a solution to Einstein's equations called the Schwarzschild metric, and this has a true horizon. However the Schwarzschild metric is time independent so it would only describe a real black hole if that black hole had existed for an infinite time and would continue to exist for an infinite time. Both of these are not possible in the real universe. So the Schwarzschild metric is only an approximate description of a real black hole, though we expect it to be a very good approximation.


Question:
Gradient tree boosting as proposed by Friedman uses decision trees as base learners. I'm wondering if we should make the base decision tree as complex as possible (fully grown) or simpler? Is there any explanation for the choice?
Random Forest is another ensemble method using decision trees as base learners.
Based on my understanding, we generally use the almost fully grown decision trees in each iteration. Am I right?

$\text{error = bias + variance}$

Boosting is based on weak learners (high bias, low variance). In
terms of decision trees, weak learners are shallow trees, sometimes
even as small as decision stumps (trees with two leaves). Boosting
reduces error mainly by reducing bias (and also to some extent variance,
by aggregating the output from many models).
On the other hand, Random Forest uses as you said fully grown
decision trees (low bias, high variance). It tackles the
error reduction task in the opposite way: by reducing variance.
The trees are made uncorrelated to maximize the decrease in variance,
but the algorithm cannot reduce bias (which is slightly higher than the
bias of an individual tree in the forest). Hence the need for large,
unpruned trees, so that the bias is initially as low as possible.

Please note that unlike Boosting (which is sequential), RF grows trees in parallel. The term iterative that you used is thus inappropriate.


Question:
Deterministic models. Clarification of the question:
The problem with these blogs is that people are inclined to start yelling at each other. (I admit, I got infected and it's difficult not to raise one's electronic voice.) I want to ask my question without an entourage of polemics.
My recent papers were greeted with scepticism. I've no problem with that. What disturbes me is the general reaction that they are "wrong". My question is summarised as follows:
Did any of these people actually read the work and can anyone tell me where a mistake was made?
Now the details. I can't help being disgusted by the "many world" interpretation, or the Bohm-de Broglie "pilot waves", and even the idea that the quantum world must be non-local is difficult to buy. I want to know what is really going on, and in order to try to get some ideas, I construct some models with various degrees of sophistication. These models are of course "wrong" in the sense that they do not describe the real world, they do not generate the Standard Model, but one can imagine starting from such simple models and adding more and more complicated details to make them look more realistic, in various stages.
Of course I know what the difficulties are when one tries to underpin QM with determinism. Simple probabilistic theories fail in an essential way. One or several of the usual assumptions made in such a deterministic theory will probably have to be abandoned; I am fully aware of that. On the other hand, our world seems to be extremely logical and natural.
Therefore, I decided to start my investigation at the other end. Make assumptions that later surely will have to be amended; make some simple models, compare these with what we know about the real world, and then modify the assumptions any way we like.
The no-go theorems tell us that a simple cellular automaton model is not likely to work. One way I tried to "amend" them, was to introduce information loss. At first sight this would carry me even further away from QM, but if you look a little more closely, you find that one still can introduce a Hilbert space, but it becomes much smaller and it may become holographic, which is something we may actually want. If you then realize that information loss makes any mapping from the deterministic model to QM states fundamentally non-local—while the physics itself stays local—then maybe the idea becomes more attractive.
Now the problem with this is that again one makes too big assumptions, and the math is quite complicated and unattractive.
So I went back to a reversible, local, deterministic automaton and asked: To what extent does this resemble QM, and where does it go wrong? With the idea in mind that we will alter the assumptions, maybe add information loss, put in an expanding universe, but all that comes later; first I want to know what goes wrong.
And here is the surprise: In a sense, nothing goes wrong. All you have to assume is that we use quantum states, even if the evolution laws themselves are deterministic. So the probability distributions are given by quantum amplitudes. The point is that, when describing the mapping between the deterministic system and the quantum system, there is a lot of freedom. If you look at any one periodic mode of the deterministic system, you can define a common contribution to the energy for all states in this mode, and this introduces a large number of arbitrary constants, so we are given much freedom.
Using this freedom I end up with quite a few models that I happen to find interesting. Starting with deterministic systems I end up with quantum systems. I mean real quantum systems, not any of those ugly concoctions. On the other hand, they are still a long way off from the Standard Model, or even anything else that shows decent, interacting particles.
Except string theory. Is the model I constructed a counterexample, showing that what everyone tells me about fundamental QM being incompatible with determinism, is wrong? No, I don't believe that. The idea was that, somewhere, I will have to modify my assumptions, but maybe the usual assumptions made in the no-go theorems will have to be looked at as well.
I personally think people are too quick in rejecting "superdeterminism". I do reject "conspiracy", but that might not be the same thing. Superdeterminism simply states that you can't "change your mind" (about which component of a spin to measure), by "free will", without also having a modification of the deterministic modes of your world in the distant past. It's obviously true in a deterministic world, and maybe this is an essential fact that has to be taken into account. It does not imply "conspiracy".
Does someone have a good, or better, idea about this approach, without name-calling? Why are some of you so strongly opinionated that it is "wrong"? Am I stepping on someone's religeous feelings? I hope not.
References:
"Relating the quantum mechanics of discrete systems to standard canonical quantum mechanics",  arXiv:1204.4926 [quant-ph];
"Duality between a deterministic cellular automaton and a bosonic quantum field theory in $1+1$ dimensions", arXiv:1205.4107 [quant-ph];
"Discreteness and Determinism in Superstrings", arXiv:1207.3612 [hep-th].

Further reactions on the answers given. (Writing this as "comment" failed, then writing this as "answer" generated objections. I'll try to erase the "answer" that I should not have put there...)
First: thank you for the elaborate answers.
I realise that my question raises philosophical issues; these are interesting and important, but not my main concern. I want to know why I find no technical problem while constructing my model. I am flattered by the impression that my theories were so "easy" to construct. Indeed, I made my presentation as transparent as possible, but it wasn't easy. There are many dead alleys, and not all models work equally well. For instance, the harmonic oscillator can be mapped onto a simple periodic automaton, but then one does hit upon technicalities: The hamiltonian of a periodic system seems to be unbounded above and below, while the harmonic oscillator has a ground state. The time-reversible cellular automaton (CA) that consists of two steps $A$ and $B$, where both $A$ and $B$ can be written as the exponent of physically reasonable Hamiltonians, itself is much more difficult to express as a Hamiltonian theory, because the BCH series does not converge. Also, explicit $3+1$ dimensional QFT models resisted my attempts to rewrite them as cellular automata. This is why I was surprised that the superstring works so nicely, it seems, but even here, to achieve this, quite a few tricks had to be invented.
@RonMaimon. I here repeat what I said in a comment, just because there the 600 character limit distorted my text too much. You gave a good exposition of the problem in earlier contributions: in a CA the "ontic" wave function of the universe can only be in specific modes of the CA. This means that the universe can only be in states $\psi_1,\ \psi_2,\ ...$ that have the property $\langle\psi_i\,|\,\psi_j\rangle=\delta_{ij}$, whereas the quantum world that we would like to describe, allows for many more states that are not at all orthonormal to each other. How could these states ever arise? I summarise, with apologies for the repetition:

We usually think that Hilbert space is separable, that is, inside every infinitesimal volume element of this world there is a Hilbert space, and the entire Hilbert space is the product of all these.
Normally, we assume that any of the states in this joint Hilbert space may represent an "ontic" state of the Universe.
I think this might not be true. The ontic states of the universe may form a much smaller class of states $\psi_i$; in terms of CA states, they must form an orthonormal set. In terms of "Standard Model" (SM) states, this orthonormal set is not separable, and this is why, locally, we think we have not only the basis elements but also all superpositions.
The orthonormal set is then easy to map back onto the CA states.

I don't think we have to talk about a non-denumerable number of states, but the number of CA states is extremely large. In short: the mathematical system allows us to choose: take all CA states, then the orthonormal set is large enough to describe all possible universes, or choose the much smaller set of SM states, then you also need many superimposed states to describe the universe. The transition from one description to the other is natural and smooth in the mathematical sense.
I suspect that, this way, one can see how a description that is not quantum mechanical at the CA level (admitting only "classical" probabilities), can "gradually" force us into accepting quantum amplitudes when turning to larger distance scales, and limiting ourselves to much lower energy levels only. You see, in words, all of this might sound crooky and vague, but in my models I think I am forced to think this way, simply by looking at the expressions: In terms of the SM states, I could easily decide to accept all quantum amplitudes, but when turning to the CA basis, I discover that superpositions are superfluous; they can be replaced by classical probabilities without changing any of the physics, because in the CA, the phase factors in the superpositions will never become observable.
@Ron I understand that what you are trying to do is something else. It is not clear to me whether you want to interpret $\delta\rho$ as a wave function. (I am not worried about the absence of $\mathrm{i}$, as long as the minus sign is allowed.) My theory is much more direct; I use the original "quantum" description with only conventional wave functions and conventional probabilities.

(New since Sunday Aug. 20, 2012)
There is a problem with my argument. (I correct some statements I had put here earlier). I have to work with two kinds of states: 1: the template states, used whever you do quantum mechanics, these allow for any kinds of superposition; and 2: the ontic states, the set of states that form the basis of the CA. The ontic states $|n\rangle$ are all orthonormal: $\langle n|m\rangle=\delta_{nm}$, so no superpositions are allowed for them (unless you want to construct a template state of course). One can then ask the question: How can it be that we (think we) see superimposed states in experiments? Aren't experiments only seeing ontic states?
My answer has always been: Who cares about that problem? Just use the rules of QM. Use the templates to do any calculation you like, compute your state $|\psi\rangle$, and then note that the CA probabilities, $\rho_n=|\langle n|\psi\rangle|^2$, evolve exactly as probabilities are supposed to do.
That works, but it leaves the question unanswered, and for some reason, my friends on this discussion page get upset by that.
So I started thinking about it. I concluded that the template states can be used to describe the ontic states, but this means that, somewhere along the line, they have to be reduced to an orthonormal set. How does this happen? In particular, how can it be that experiments strongly suggest that superpositions play extremely important roles, while according to my theory, somehow, these are plutoed by saying that they aren't ontic?
Looking at the math expressions, I now tend to think that orthonormality is restored by "superdeterminism", combined with vacuum fluctuations. The thing we call vacuum state, $|\emptyset\rangle$, is not an ontological state, but a superposition of many, perhaps all, CA states. The phases can be chosen to be anything, but it makes sense to choose them to be $+1$ for the vacuum. This is actually a nice way to define phases: all other phases you might introduce for non-vacuum states now have a definite meaning.
The states we normally consider in an experiment are usually orthogonal to the vacuum. If we say that we can do experiments with two states, $A$ and $B$, that are not orthonormal to each other, this means that these are template states; it is easy to construct such states and to calculate how they evolve. However, it is safe to assume that, actually, the ontological states $|n\rangle$ with non-vanishing inner product with $A$, must be different from the states $|m\rangle$ that occur in $B$, so that, in spite of the template, $\langle A|B\rangle=0$. This is because the universe never repeats itself exactly. My physical interpretation of this is "superdeterminism": If, in an EPR or Bell experiment, Alice (or Bob) changes her (his) mind about what to measure, she (he) works with states $m$ which all differ from all states $n$ used previously. In the template states, all one has to do is assume at least one change in one of the physical states somewhere else in the universe. The contradiction then disappears.
The role of vacuum fluctuations is also unavoidable when considering the decay of an unstable particle.
I think there's no problem with the above arguments, but some people find it difficult to accept that the working of their minds may have any effect at all on vacuum fluctuations, or the converse, that vacuum fluctuations might affect their minds. The "free will" of an observer is at risk; people won't like that.
But most disturbingly, this argument would imply that what my friends have been teaching at Harvard and other places, for many decades as we are told, is actually incorrect. I want to stay modest; I find this disturbing.
A revised version of my latest paper was now sent to the arXiv (will probably be available from Monday or Tuesday). Thanks to you all. My conclusion did not change, but I now have more precise arguments concerning Bell's inequalities and what vacuum fluctuations can do to them.

I can tell you why I don't believe in it. I think my reasons are different from most physicists' reasons, however.
Regular quantum mechanics implies the existence of quantum computation. If you believe in the difficulty of factoring (and a number of other classical problems), then a deterministic underpinning for quantum mechanics would seem to imply one of the following.

There is a classical polynomial-time algorithm for factoring and other problems which can be solved on a quantum computer.
The deterministic underpinnings of quantum mechanics require $2^n$ resources for a system of size $O(n)$.
Quantum computation doesn't actually work in practice.

None of these seem at all likely to me. For the first, it is quite conceivable that there is a polynomial-time algorithm for factoring, but quantum computation can solve lots of similar periodicity problems, and you can argue that there can't be a single algorithm that solves all of them on a classical computer, so you would have to have different classical algorithms for each classical problem that a quantum computer can solve by period finding.
For the second, deterministic underpinnings of quantum mechanics that require $2^n$ resources for a system of size $O(n)$ are really unsatisfactory (but maybe quite possible ... after all, the theory that the universe is a simulation on a classical computer falls in this class of theories, and while truly unsatisfactory, can't be ruled out by this argument).
For the third, I haven't seen any reasonable way to how you could make quantum computation impossible while still maintaining consistency with current experimental results.


Question:
A famous exercise which one encounters while doing Complex Analysis (Residue theory) is to prove that the given integral:
$$\int\limits_0^\infty \frac{\sin x} x \,\mathrm  dx = \frac \pi 2$$
Well, can anyone prove this without using Residue theory?  I actually thought of using the series representation of $\sin x$:
$$\int\limits_0^\infty \frac{\sin x} x \, dx = \lim\limits_{n \to \infty} \int\limits_0^n \frac{1}{t} \left( t - \frac{t^3}{3!} + \frac{t^5}{5!} + \cdots \right) \,\mathrm  dt$$
but I don't see how $\pi$ comes here, since we need the answer to be equal to $\dfrac{\pi}{2}$.

I believe this can also be solved using double integrals.
It is possible (if I remember correctly) to justify switching the order of integration to give the equality:
$$\int_{0}^{\infty} \Bigg(\int_{0}^{\infty} e^{-xy} \sin x \,dy \Bigg)\, dx = \int_{0}^{\infty} \Bigg(\int_{0}^{\infty} e^{-xy} \sin x \,dx \Bigg)\,dy$$
Notice that
$$\int_{0}^{\infty} e^{-xy} \sin x\,dy = \frac{\sin x}{x}$$
This leads us to
$$\int_{0}^{\infty} \Big(\frac{\sin x}{x} \Big) \,dx = \int_{0}^{\infty} \Bigg(\int_{0}^{\infty} e^{-xy} \sin x \,dx \Bigg)\,dy$$
Now the right hand side can be found easily, using integration by parts.
$$\begin{align*}
I &= \int e^{-xy} \sin x \,dx = -e^{-xy}{\cos x} - y \int e^{-xy} \cos x \, dx\\
&= -e^{-xy}{\cos x} - y \Big(e^{-xy}\sin x + y \int e^{-xy} \sin x \,dx \Big)\\
&= \frac{-ye^{-xy}\sin x - e^{-xy}\cos x}{1+y^2}.
\end{align*}$$
Thus $$\int_{0}^{\infty} e^{-xy} \sin x \,dx = \frac{1}{1+y^2}$$
Thus $$\int_{0}^{\infty} \Big(\frac{\sin x}{x} \Big) \,dx = \int_{0}^{\infty}\frac{1}{1+y^2}\,dy = \frac{\pi}{2}.$$


Question:
What's the similarities and differences between these 3 methods:

Bagging,
Boosting,
Stacking?

Which is the best one? And why?
Can you give me an example for each?

All three are so-called "meta-algorithms": approaches to combine several machine learning techniques into one predictive model in order to decrease the variance (bagging), bias (boosting) or improving the predictive force (stacking alias ensemble).
Every algorithm consists of two steps:

Producing a distribution of simple ML models on subsets of the original data.
Combining the distribution into one "aggregated" model.

Here is a short description of all three methods:

Bagging (stands for Bootstrap Aggregating) is a way to decrease the variance of your prediction by generating additional data for training from your original dataset using combinations with repetitions to produce multisets of the same cardinality/size as your original data. By increasing the size of your training set you can't improve the model predictive force, but just decrease the variance, narrowly tuning the prediction to expected outcome.
Boosting is a two-step approach, where one first uses subsets of the original data to produce a series of averagely performing models and then "boosts" their performance by combining them together using a particular cost function (=majority vote). Unlike bagging, in the classical boosting the subset creation is not random and depends upon the performance of the previous models: every new subsets contains the elements that were (likely to be) misclassified by previous models.
Stacking is a similar to boosting: you also apply several models to your original data. The difference here is, however, that you don't have just an empirical formula for your weight function, rather you introduce a meta-level and use another model/approach to estimate the input together with outputs of every model to estimate the weights or, in other words, to determine what models perform well and what badly given these input data.

Here is a comparison table:

As you see, these all are different approaches to combine several models into a better one, and there is no single winner here: everything depends upon your domain and what you're going to do. You can still treat stacking as a sort of more advances boosting, however, the difficulty of finding a good approach for your meta-level makes it difficult to apply this approach in practice.
Short examples of each:

Bagging: Ozone data.
Boosting: is used to improve optical character recognition (OCR) accuracy.
Stacking: is used in classification of cancer microarrays in medicine.


Question: What differences or other criteria can be used to help decide between using overlap-add and overlap-save for filtering?  Both overlap-add and overlap-save are described as algorithms for doing FFT based fast convolution of data streams with FIR filter kernels.  What are the latency, computational efficiency or caching locality (etc.) differences, if any?  Or are they the same?

Essentially, OS is slightly more efficient since it does not require the addition of the overlapping transients.  However, you may want to use OA if you need to reuse the FFTs with zero-padding rather than repeated samples.
Here is a quick overview from an article I wrote a while ago

Fast convolution refers to the blockwise use of circular convolution
to accomplish linear convolution. Fast convolution can be accomplished
by OA or OS methods. OS is also known as “overlap- scrap” . In OA
filtering, each signal data block contains only as many samples as
allows circular convolution to be equivalent to linear convolution.
The signal data block is zero-padded prior to the FFT to prevent the
filter impulse response from “wrapping around” the end of the
sequence. OA filtering adds the input-on transient from one block with
the input-off transient from the previous block. In OS filtering,
shown in Figure 1, no zero-padding is performed on the input data,
thus the circular convolution is not equivalent to linear convolution.
The portions that “wrap around” are useless and discarded. To
compensate for this, the last part of the previous input block is used
as the beginning of the next block. OS requires no addition of
transients, making it faster than OA.


Question:
In my last question I asked why we don't see increased complexity in artificial life simulations of evolution. It seems I had fallen for a common misconception, that evolution was about improvement by increasing complexity. One comment discussing that post read

"... he [David Deutsch] is falling for one of the biggest
  misconceptions about evolution that you can, that evolution is about
  improvement. Evolution has simply only ever been about change..."

However, when you look at the history of life you see increases in complexity. You see this increasing complexity evolving over billions of years, suggesting that it requires an explanation.
My question
If evolution is not about increasing complexity then how does so much complexity evolve?

I think possibly the problem here is the way you're approaching the issue.
You're considering improvement as anything that increases the abilities or complexity of the organism—that isn't necessarily what an improvement is though. The outcome of natural selection is that the organism best equipped to survive/reproduce in a certain environment is the most successful. So, for example, thermophillic archaea do much better in 60°C-plus pools of water than humans do. Our capacity to process information, use tools, etc. doesn't actually confer much advantage in that situation. And there can be downsides to that kind of complexity as well, requiring more energy and longer developmental periods. So, natural selection in 60°C-plus pools of water gives you archaea, and in (presumably) the plains of East Africa, it gives you humans.
The comment you quote mentions sickle-cell anaemia, which is a different example. While there is little benefit to having the sickle-cell anaemia allele in a temperate region, in those regions where malaria is endemic, heterozygosity can provide a survival advantage, and so the allele is maintained in the population. If you're someone living in a malaria-endemic region, and you don't have access to antimalarials, heterozygosity for the sickle-cell anaemia allele is arguably an improvement. It depends entirely on how you define the word.
The fundamental principal of natural selection is that it favours the organism most suited to a particular environment. But, that isn't always the most complex organism. It's important not to confuse human-like with better. It isn't the universal endpoint of evolution to produce an organism similar to us, just the organism most suited to the environment in question.
Also, to briefly address the previous question you asked—you asserted that we must be missing something from the process of evolution because we were unable to simulate it. You also pointed out that (in your opinion) we have sufficient computing power to simulate the kinds of organisms you're referring to. But natural selection is intrinsically linked to the environment it occurs in, so the simulation wouldn't just have to accurately simulate the biological processes of the organism, but also all of the external pressures the organism faces. I'd imagine that, in simulating evolution, that would be the real obstacle.


Question:
LIGO has announced the detection of gravitational waves on 11 Feb, 2016. I was wondering why the detection of gravitational waves was so significant?
I know it is another confirmation of general relativity (GR), but I thought we had already confirmed GR beyond much doubt. What extra stuff would finding gravitational waves teach us? Is the detection of gravitational waves significant in and of itself, or is there data which can be extracted from the waves which will be more useful?

Gravitational waves are qualitatively different from other detections.
As much as we have tested GR before, it's still reassuring to find a completely different test that works just as well. The most notable tests so far have been the shifting of Mercury's orbit, the correct deflection of light by massive objects, and the redshifting of light moving against gravity. In these cases, spacetime is taken to be static (unchanging in time, with no time-space cross terms in the metric). Gravitational waves, on the other hand, involve a time-varying spacetime.
Gravitational waves provide a probe of strong-field gravity.
The tests so far have all been done in weak situations, where you have to measure things pretty closely to see the difference between GR and Newtonian gravity. While gravitational waves themselves are a prediction of linearized gravity and are the very essence of small perturbations, their sources are going to be very extreme environments -- merging black holes, exploding stars, etc. Now a lot of things can go wrong between our models of these extreme phenomena and our recording of a gravitational wave signal, but if the signal agrees with our predictions, that's a sign that not only are we right about the waves themselves, but also about the sources.
Gravitational waves are a new frontier in astrophysics.
This point is often forgotten when we get so distracted with just finding any signal. Finding the first gravitational waves is only the beginning for astronomical observations.
With just two detectors, LIGO for instance cannot pinpoint sources on the sky any better than "somewhere out there, roughly." Eventually, as more detectors come online, the hope is to be able to localize signals better, so we can simultaneously observe electromagnetic counterparts. That is, if the event causing the waves is the merger of two neutron stars, one might expect there to be plenty of light released as well. By combining both types of information, we can gain quite a bit more knowledge about the system.
Gravitational waves are also good at probing the physics at the innermost, most-obscured regions in cataclysmic events. For most explosions in space, all we see now is the afterglow -- the hot, radioactive shell of material left behind -- and we can only infer indirectly what processes were happening at the core. Gravitational waves provide a new way to gain insight in this respect.


Question:
I was told in my organic chemistry course that $\text{S}_\text{N}1$ and $\text{S}_\text{N}2$ reactions did not occur at $\text{sp}^2$ centres. When I asked why, I was not given a satisfactory explanation. For $\text{S}_\text{N}2$ it was suggested that the reaction could not proceed with inversion of configuration, as this would disrupt the orbital overlap causing the $\pi$ bond. I couldn't find any explanation in Clayden's Organic Chemistry (2nd ed.), or even a mention of it.
Could anyone please explain why? I have attached some curly arrow diagrams that seem reasonable enough to me, although I realise that being able to draw a reaction mechanism does not mean it is valid.

Sometimes, especially in introductory courses the instructor will try to keep things "focused" in order to promote learning.  Still, it's unfortunate that the instructor couldn't respond in a more positive and stimulating way to your question.
These reactions do occur at $\ce{sp^2}$ hybridized carbon atoms, they are often just energetically more costly, and therefore somewhat less common. Consider when a nucleophile reacts with a carbonyl compound, the nucleophile attacks the carbonyl carbon atom in an $\ce{S_{N}2}$ manner.  The electrons in the C-O $\pi$–bond can be considered as the leaving group and a tetrahedral intermediate is formed with a negative charge on oxygen. It is harder to do this with a carbon-carbon double bond (energetically more costly) because you would wind up with a negative charge on carbon (instead of oxygen), which is energetically less desirable (because of the relative electronegativities of carbon and oxygen).

If you look at the Michael addition reaction, the 1,4-addition of a nucleophile to the carbon-carbon double bond in an $\ce{\alpha-\beta}$ unsaturated carbonyl system, this could be viewed as an $\ce{S_{N}2}$ attack on a carbon-carbon double bond, but again, it is favored (lower in energy) because you create an intermediate with a negative charge on oxygen.

$\ce{S_{N}1}$ reactions at $\ce{sp^2}$ carbon are well documented.  Solvolysis of vinyl halides in very acidic media is an example.  The resultant vinylic carbocations are actually stable enough to be observed using nmr spectroscopy.  The picture below helps explain why this reaction is so much more difficult (energetically more costly) than the more common solvolysis of an alkyl halide.  In the solvolysis of the alkyl halide we produce a traditional carbocation with an empty p orbital.  In the solvolysis of the vinyl halide we produce a carbocation with the positive charge residing in an $\ce{sp^2}$ orbital. Placing positive charge in an $\ce{sp^2}$ orbital is a higher energy situation compared to placing it in a p orbital (electrons prefer to be in orbitals with higher s density, it stabilizes them because the more s character in an orbital the lower its energy; conversely, in the absence of electrons, an orbital prefers to have high p character and mix the remaining s character into other bonding orbitals that do contain electrons in order to lower their energy).


Question:
There are a lot of poorly drawn schematics here. A few times people have actually asked for critiques of their schematics. This question is intended as a single repository on schematic drawing rules and guidelines can point people to. The question is
What are the rules and guidelines for drawing good schematics?

Note: This is about schematics themselves, not about the circuits they represent.

A schematic is a visual representation of a circuit.  As such, its
purpose is to communicate a circuit to someone else.  A schematic in a
special computer program for that purpose is also a machine-readable
description of the circuit. This use is easy to judge in absolute terms.
Either the proper formal rules for describing the circuit are followed and
the circuit is correctly defined or it isn't. Since there are hard rules
for that and the result can be judged by machine, this isn't the point of
the discussion here.  This discussion is about rules, guidelines, and
suggestions for good schematics for the first purpose, which is to
communicate a circuit to a human.  Good and bad will be
judged here in that context.
Since a schematic is to communicate information, a good schematic does
this quickly, clearly, and with a low chance of misunderstanding.  It is
necessary but far from sufficient for a schematic to be correct.  If a
schematic is likely to mislead a human observer, it is a bad schematic
whether you can eventually show that after due deciphering it was in fact
correct.  The point is clarity. A technically correct but
obfuscated schematic is still a bad schematic.
Some people have their own silly-ass opinions, but here are the rules
(actually, you'll probably notice broad agreement between experienced
people on most of the important points):

Use component designators
This is pretty much automatic with any schematic capture program, but we still often see schematics here without them.  If you draw your schematic on a napkin and then scan it, make sure to add component designators. These make the circuit much easier to talk about.  I have
skipped over questions when schematics didn't have component designators
because I didn't feel like bothering with the second 10 kΩ
resistor from the left by the top pushbutton.  It's a lot easier to
say R1, R5, Q7, etc.
Clean up text placement
Schematic programs generally plunk down part names and values based on a generic part definition.  This means they often end up in
inconvenient places in the schematic when other parts are placed nearby.
Fix it.  That's part of the job of drawing a schematic.  Some schematic capture programs make this easier than others.  In Eagle for example,
unfortunately, there can only be one symbol for a part. Some parts are commonly placed in different orientations, horizontal and vertical in the case of resistors for example.  Diodes can be placed in at least 4
orientations since they have direction too.  The placement of text around a part, like the component designator and value, probably won't work in other orientations than it was originally drawn in.  If you rotate a stock part, move the text around afterward so that it is easily readable, clearly belongs to that part, and doesn't collide with other parts of the drawing.  Vertical text looks stupid and makes the schematic hard to read.
I make separate redundant parts in Eagle that differ only in the symbol orientation and therefore the text placement.  That's more work upfront but makes it easier when drawing a schematic.  However, it doesn't matter how you achieve a neat and clear end result, only that you do.  There is no excuse.  Sometimes we hear whines like " But
CircuitBarf 0.1 doesn't let me do that".  So get something that does.  Besides, CircuitBarf 0.1 probably does let you do it, just that you were too lazy to read the manual to learn how and too sloppy to care. Draw it (neatly!) on paper and scan it if you have to.  Again,
there is no excuse.
For example, here are some parts at different orientations.  Note how
the text is in different places relative to parts to make things neat
and clear.

Don't let this happen to you:

Yes, this is actually a small snippet of what someone dumped on us
here.
Basic layout and flow
In general, it is good to put higher voltages towards the top, lower
voltages towards the bottom and logical flow left to right.  That's
clearly not possible all the time, but at least a generally higher level
effort to do this will greatly illuminate the circuit to those reading
your schematic.
One notable exception to this is feedback signals.  By their very
nature, they feed "back" from downstream to upstream, so they
should be shown sending information opposite of the main flow.
Power connections should go up to positive voltages and down to negative voltages.  Don't do this:

There wasn't room to show the line going down to ground because other stuff was already there.  Move it.  You made the mess, you can unmake
it.  There is always a way.
Following these rules causes common subcircuits to be drawn similarly most of the time.  Once you get more experience looking at schematics,
these will pop out at you and you will appreciate this.  If stuff is drawn every which way, then these common circuits will look visually different every time and it will take others longer to understand your schematic.  What's this mess, for example?

After some deciphering, you realize "Oh, it's a common emitter amplifier.  Why didn't that #%&^$@#$% just draw it like one in the first
place!?":

Draw pins according to function
Show pins of ICs in a position relevant to their function, NOT HOW THEY
HAPPEN TO STICK OUT OF THE CHIP.  Try to put positive power pins at the top,
negative power pins (usually grounds) at the bottom, inputs at left, and outputs at right.  Note that this fits with the general schematic layout as described above. Of course, this isn't always reasonable and possible.
General-purpose parts like microcontrollers and FPGAs have pins that can
be input and output depending on use and can even vary at run time.  At
least you can put the dedicated power and ground pins at top and bottom,
and possibly group together any closely related pins with dedicated
functions, like crystal driver connections.
ICs with pins in physical pin order are difficult to understand. Some people use the excuse that this aids in debugging, but with a little thought you can see that's not true.  When you want to look at something
with a scope, which question is more common "I want to look at the
clock, what pin is that?" or "I want to look at pin 5, what
function is that?". In some rare cases, you might want to go around a
IC and look at all the pins, but the first question is by far more common.
Physical pin order layouts obfuscate the circuit and make debugging more difficult.  Don't do it.
Direct connections, within reason
Spend some time with placement reducing wire crossings and the like.
The recurring theme here is clarity.  Of course, drawing a direct connection line isn't always possible or reasonable. Obviously, it can't
be done with multiple sheets, and a messy rats nest of wires is worse
than a few carefully chosen "air wires".
It is impossible to come up with a universal rule here, but if you
constantly think of the mythical person looking over your shoulder
trying to understand the circuit from the schematic you are drawing,
you'll probably do alright. You should be trying to help people
understand the circuit easily, not make them figure it out despite the
schematic.
Design for regular size paper
The days of electrical engineers having drafting tables and being set up to work with D size drawings are long gone.  Most people only have access to regular page-size printers, like for 8 1/2 x 11-inch paper here in the US.  The exact size is a little different all around the world, but they are all roughly what you can easily hold in front of you or place on your desk. There is a reason this size evolved as a
standard. Handling larger paper is a hassle.  There isn't room on the desk, it ends up overlapping the keyboard, pushes things off your desk when you move it, etc.
The point is to design your schematic so that individual sheets are nicely readable on a single normal page, and on the screen at about the same size.  Currently, the largest common screen size is 1920 x 1080.
Having to scroll a page at that resolution to see necessary detail is
annoying.
If that means using more pages, go ahead.  You can flip pages back and forth with a single button press in Acrobat Reader.  Flipping pages
is preferable to panning a large drawing or dealing with outsized paper.
I also find that one normal page at reasonable detail is a good size to show a subcircuit.  Think of pages in schematics like paragraphs in a
narrative.  Breaking a schematic into individually labeled sections by pages can actually help readability if done right.  For example, you might have a page for the power input section, the immediate microcontroller connections, the analog inputs, the H bridge drive power outputs, the ethernet interface, etc.  It's actually useful to break up the schematic this way even if it had nothing to do with drawing size.
Here is a small section of a schematic I received.  This is from a
screenshot displaying a single page of the schematic maximized in
Acrobat Reader on a 1920 x 1200 screen.

In this case, I was being paid in part to look at this schematic so I
put up with it, although I probably used more time and therefore charged
the customer more money than if the schematic had been easier to work
with.  If this was from someone looking for free help like on this web
the site, I would have thought to myself screw this and gone on to
answer someone else's question.
Label key nets
Schematic capture programs generally let you give nets nicely readable names. All nets probably have names inside the software, just
that they default to some gobbledygook unless you explicitly set them.
If a net is broken up into visually unconnected segments, then you absolutely have to let people know the two seemingly disconnected nets are really the same.  Different packages have different built-in ways to show that.  Use whatever works with the software you have, but in any case, give the net a name and show that name at each separately drawn segment.  Think of that as the lowest common denominator or using "air wires" in a schematic.  If your software supports it and you think it helps with clarity, by all means, use little "jump point" markers or whatever.  Sometimes these even give you the sheet and coordinates of one or more corresponding jump points.  That's all great but label any such net anyway.
The important point is that the little name strings for these nets
are derived automatically from the internal net name by the software.
Never draw them manually as arbitrary text that the software doesn't understand as the net name. If separate sections of the net ever get disconnected or separately renamed by accident, the software will automatically show this since the name shown comes from the actual net name, not something you type in separately.  This is a lot like a
variable in a computer language. You know that multiple uses of the variable symbol refer to the same variable.
Another good reason for net names is short comments.  I sometimes name and then show the names of nets only to give a quick idea what the purpose of that net is.  For example, seeing that a net is called "5V"
or "MISO" could help a lot in understanding the circuit.  Many short nets don't need a name or clarification, and adding names would hurt more due to clutter than they would illuminate.  Again, the whole point is clarity.  Show a meaningful net name when it helps in understanding the circuit, and don't when it would be more distracting than useful.
Keep names reasonably short
Just because your software lets you enter 32 or 64 character net names, doesn't mean you should.  Again, the point is about clarity.  No names
is no information, but lots of long names are clutter, which then
decreases clarity.  Somewhere in between is a good tradeoff.  Don't get
silly and write "8 MHz clock to my PIC", when simply "CLOCK", "CLK", or
"8MHZ" would convey the same information.
See this ANSI/IEEE standard for recommended pin name abbreviations.
Upper case symbol names
Use all caps for net names and pin names.  Pin names are almost always shown upper case in datasheets and schematics.  Various schematic programs, Eagle included, don't even allow for lower case names.  One advantage of this, which is also helped when the names aren't too long,
is that they stick out in the regular text.  If you do write real comments in the schematic, always write them in mixed case but make sure to upper case symbol names to make it clear they are symbol names and not part of your narrative.  For example, "The input signal TEST1 goes high to
turn on Q1, which resets the processor by driving MCLR low.".  In this case, it is obvious that TEST1, Q1, and MCLR refer to names in the schematic and aren't part of the words you are using in the description.
Show decoupling caps by the part
Decoupling caps must be physically close to the part they are decoupling due to their purpose and basic physics.  Show them that way.
Sometimes I've seen schematics with a bunch of decoupling caps off in a
corner.  Of course, these can be placed anywhere in the layout, but by placing them by their IC you at least show the intent of each cap.  This makes it much easier to see that proper decoupling was at
least thought about, more likely a mistake is caught in a design review,
and more likely the cap actually ends up where intended when the layout is
done.
Dots connect, crosses don't
Draw a dot at every junction.  That's the convention.  Don't be lazy.
Any competent software will enforce this any way, but surprisingly we still see schematics without junction dots here occasionally.  It's a rule. We don't care whether you think it's silly or not.  That's how it's done.
Sort of related, try to keep junctions to Ts, not 4-way crosses.  This isn't as hard a rule, but stuff happens.  With two lines crossing, one vertical the other horizontal, the only way to know whether they are connected is whether the little junction dot is present.  In past days when schematics were routinely photocopied or otherwise optically reproduced, junction dots could disappear after a few generations, or could sometimes even appear at crosses when they weren't there originally.  This is less important now that schematics are generally in a computer, but it's not a bad idea to be extra careful.  The way to do
that is to never have a 4-way junction.
If two lines cross, then they are never connected, even if after some reproduction or compression artifacts it looks like there maybe is a dot there.  Ideally connections or crossovers would be unambiguous without junction dots, but in reality, you want as little chance of misunderstanding as possible.  Make all junctions Ts with dots, and all
crossing lines are therefore different nets without dots.

Look back and you can see the point of all these rules is to make it as
easy as possible for someone else to understand the circuit from the
schematic, and to maximize the chance that understanding is correct.

Good schematics show you the circuit.  Bad schematics make you
decipher them.

There is another human point to this too.  A sloppy schematic
shows lack of attention to detail and is irritating and insulting to anyone
you ask to look at it.  Think about it.  It says to others "Your
aggravation with this schematic isn't worth my time to clean it up"
which is basically saying "I'm more important than you are".  That's
not a smart thing to say in many cases, like when you are asking for free
help here, showing your schematic to a customer, teacher, etc.
Neatness and presentation count.  A lot.  You are judged
by your presentation quality every time you present something, whether you
think that's how it should be or not.  In most cases, people won't bother
to tell you either. They'll just go on to answer a
different question, not look for some good points that might make the
grade one notch higher, or hire someone else, etc.  When you give someone a sloppy schematic (or
any other sloppy work from you), the first thing they're going to think is
"What a jerk".  Everything else they think of you and your work
will be colored by that initial impression.  Don't be that loser.


Question:
How is it that you read a mathematics book?
Do you keep a notebook of definitions? What about theorems?
Do you do all the exercises? Focus on or ignore the proofs?
I have been reading Munkres, Artin, Halmos, etc. but I get a bit lost usually around the middle. Also, about how fast should you be reading it? Any advice is wanted, I just reached the upper division level.

This method has worked well for me (but what works well for one person won't necessarily work well for everyone).  I take it in several passes:
Read 0: Don't read the book, read the Wikipedia article or ask a friend what the subject is about.  Learn about the big questions asked in the subject, and the basics of the theorems that answer them.  Often the most important ideas are those that can be stated concisely, so you should be able to remember them once you are engaging the book.
Read 1: Let your eyes jump from definition to lemma to theorem without reading the proofs in between unless something grabs your attention or bothers you. If the book has exercises, see if you can do the first one of each chapter or section as you go.
Read 2: Read the book but this time read the proofs.  But don't worry if you don't get all the details.  If some logical jump doesn't make complete sense, feel free to ignore it at your discretion as long as you understand the overall flow of reasoning.
Read 3: Read through the lens of a skeptic.  Work through all of the proofs with a fine toothed comb, and ask yourself every question you think of.  You should never have to ask yourself "why" you are proving what you are proving at this point, but you have a chance to get the details down.
This approach is well suited to many math textbooks, which seem to be written to read well to people who already understand the subject.  Most of the "classic" textbooks are labeled as such because they are comprehensive or well organized, not because they present challenging abstract ideas well to the uninitiated.
(Steps 1-3 are based on a three step heuristic method for writing proofs: convince yourself, convince a friend, convince a skeptic)


Question:
I'm learning how to apply the VSEPR theory to Lewis structures and in my homework, I'm being asked to provide the hybridization of the central atom in each Lewis structure I've drawn.
I've drawn out the Lewis structure for all the required compounds and figured out the arrangements of the electron regions, and figured out the shape of each molecule. I'm being asked to figure out the hybridization of the central atom of various molecules.
I found a sample question with all the answers filled out:
$\ce{NH3}$
It is $\mathrm{sp^3}$ hybridized.
Where does this come from? I understand how to figure out the standard orbitals for an atom, but I'm lost with hybridization.
My textbook uses $\ce{CH4}$ as an example.
Carbon has $\mathrm{2s^2 \,2p^2}$, but in this molecule, it has four $\mathrm{sp^3}$. I understand the purpose of four (there are four hydrogens), but where did the "3" in $\mathrm{sp^3}$ come from?
How would I figure out something more complicated like $\ce{H2CO}$?

If you can assign the total electron geometry (geometry of all electron domains, not just bonding domains) on the central atom using VSEPR, then you can always automatically assign hybridization. Hybridization was invented to make quantum mechanical bonding theories work better with known empirical geometries. If you know one, then you always know the other.

Linear - $\ce{sp}$ - the hybridization of one $\ce{s}$ and one $\ce{p}$ orbital produce two hybrid orbitals oriented $180^\circ$ apart.
Trigonal planar - $\ce{sp^2}$ - the hybridization of one $\ce{s}$ and two $\ce{p}$ orbitals produce three hybrid orbitals oriented $120^\circ$ from each other all in the same plane.
Tetrahedral - $\ce{sp^3}$ - the hybridization of one $\ce{s}$ and three $\ce{p}$ orbitals produce four hybrid orbitals oriented toward the points of a regular tetrahedron, $109.5^\circ$ apart.
Trigonal bipyramidal - $\ce{dsp^3}$ or $\ce{sp^3d}$ - the hybridization of one $\ce{s}$, three $\ce{p}$, and one $\ce{d}$ orbitals produce five hybrid orbitals oriented in this weird shape: three equatorial hybrid orbitals oriented $120^\circ$ from each other all in the same plane and two axial orbitals oriented $180^\circ$ apart, orthogonal to the equatorial orbitals.
Octahedral - $\ce{d^2sp^3}$ or $\ce{sp^3d^2}$ - the hybridization of one $\ce{s}$, three $\ce{p}$, and two $\ce{d}$ orbitals produce six hybrid orbitals oriented toward the points of a regular octahedron $90^\circ$ apart.

I assume you haven't learned any of the geometries above steric number 6 (since they are rare), but they each correspond to a specific hybridization also.

$\ce{NH3}$

For $\ce{NH3}$, which category does it fit in above? Remember to count the lone pair as an electron domain for determining total electron geometry. Since the sample question says $\ce{NH3}$ is $\ce{sp^3}$, then $\ce{NH3}$ must be tetrahedral. Make sure you can figure out how $\ce{NH3}$ has tetrahedral electron geometry.

For $\ce{H2CO}$

Start by drawing the Lewis structure. The least electronegative atom that is not a hydrogen goes in the center (unless you have been given structural arrangement).
Determine the number of electron domains on the central atom.
Determine the electron geometry using VSEPR. Correlate the geometry with the hybridization.
Practice until you can do this quickly.


Question:
I know that generative means "based on $P(x,y)$" and discriminative means "based on $P(y|x)$," but I'm confused on several points:

Wikipedia (+ many other hits on the web) classify things like SVMs and decision trees as being discriminative. But these don't even have probabilistic interpretations. What does discriminative mean here? Has discriminative just come to mean anything that isn't generative?
Naive Bayes (NB) is generative because it captures $P(x|y)$ and $P(y)$, and thus you have $P(x,y)$ (as well as $P(y|x)$). Isn't it trivial to make, say, logistic regression (the poster boy of discriminative models) "generative" by simply computing $P(x)$ in a similar fashion (same independence assumption as NB, such that $P(x) = P(x_0) P(x_1) ... P(x_d)$, where the MLE for $P(x_i)$ are just frequencies)?
I know that discriminative models tend to outperform generative ones. What's the practical use of working with generative models? Being able to generate/simulate data is cited, but when does this come up? I personally only have experience with regression, classification, collab. filtering over structured data, so are the uses irrelevant to me here? The "missing data" argument ($P(x_i|y)$ for missing $x_i$) seems to only give you an edge with training data (when you actually know $y$ and don't need to marginalize over $P(y)$ to get the relatively dumb $P(x_i)$ which you could've estimated directly anyway), and even then imputation is much more flexible (can predict based not just on $y$ but other $x_i$'s as well).
What's with the completely contradictory quotes from Wikipedia? "generative models are typically more flexible than discriminative models in expressing dependencies in complex learning tasks" vs. "discriminative models can generally express more complex relationships between the observed and target variables"

Related question that got me thinking about this.

The fundamental difference between discriminative models and generative models is:

Discriminative models learn the (hard or soft) boundary between classes
Generative models model the distribution of individual classes

To answer your direct questions:

SVMs (Support Vector Machines) and DTs (Decision Trees) are discriminative because they learn explicit boundaries between classes.  SVM is a maximal margin classifier, meaning that it learns a decision boundary that maximizes the distance between samples of the two classes, given a kernel.  The distance between a sample and the learned decision boundary can be used to make the SVM a "soft" classifier.  DTs learn the decision boundary by recursively partitioning the space in a manner that maximizes the information gain (or another criterion).

It is possible to make a generative form of logistic regression in this manner.  Note that you are not using the full generative model to make classification decisions, though.

There are a number of advantages generative models may offer, depending on the application.  Say you are dealing with non-stationary distributions, where the online test data may be generated by different underlying distributions than the training data.  It is typically more straightforward to detect distribution changes and update a generative model accordingly than do this for a decision boundary in an SVM, especially if the online updates need to be unsupervised.  Discriminative models also do not generally function for outlier detection, though generative models generally do.  What's best for a specific application should, of course, be evaluated based on the application.

(This quote is convoluted, but this is what I think it's trying to say) Generative models are typically specified as probabilistic graphical models, which offer rich representations of the independence relations in the dataset.  Discriminative models do not offer such clear representations of relations between features and classes in the dataset.  Instead of using resources to fully model each class, they focus on richly modeling the boundary between classes.  Given the same amount of capacity (say, bits in a computer program executing the model), a discriminative model thus may yield more complex representations of this boundary than a generative model.