Question:
edit: Results are current as of Dec 4, 2018 13:00 PST.
Background
K-mers have many uses in bioinformatics, and for this reason it would be useful to know the most RAM-efficient and fastest way to work with them programmatically. There have been questions covering what canonical k-mers are, how much RAM k-mer storage theoretically takes, but we have not yet looked at the best data structure to store and access k-mers and associated values with.
Question
What data structure in C++ simultaneously allows the most compact k-mer storage, a property, and the fastest lookup time? For this question I choose C++ for speed, ease-of-implementation, and access to lower-level language features if desired. Answers in other languages are acceptable, too.
Setup

For benchmarking:

I propose to use a standard fasta file for everyone to use. This program, generate-fasta.cpp, generates two million sequences ranging in length between 29 and 300, with a peak of sequences around length 60.
Let's use k=29 for the analysis, but ignore implementations that require knowledge of the k-mer size before implementation. Doing so will make the resulting data structure more amenable to downstream users who may need other sizes k.
Let's just store the most recent read that the k-mer appeared in as the property to retrieve during k-mer lookup. In most applications it is important to attach some value to each k-mer such as a taxon, its count in a dataset, et cetera.
If possible, use the string parser in the code below for consistency between answers.
The algorithm should use canonical k-mers. That is, a k-mer and its reverse complement are considered to be the same k-mer.

Here is generate-fasta.cpp. I used the command g++ generate_fasta.cpp -o generate_fasta to compile and the command ./generate_fasta > my.fasta to run it:
return 0;
//generate a fasta file to count k-mers
#include <iostream>
#include <random>

char gen_base(int q){
  if (q <= 30){
    return 'A';
  } else if ((q > 30) && (q <=60) ){
    return 'T';
  } else if ((q > 60) && (q <=80) ){
    return 'C';
  } else if (q > 80){
    return 'G';
  }
  return 'N';
}

int main() {
  unsigned seed = 1;
  std::default_random_engine generator (seed);
  std::poisson_distribution<int> poisson (59);
  std::geometric_distribution<int> geo (0.05);
  std::uniform_int_distribution<int> uniform (1,100);
  int printval;
  int i=0;
  while(i<2000000){
    if (i % 2 == 0){
      printval = poisson(generator);
    } else {
      printval = geo(generator) + 29;
    }
    if (printval >= 29){
      std::cout << '>' << i << '\n';
      //std::cout << printval << '\n';
      for (int j = 0; j < printval; j++){
        std::cout << gen_base(uniform(generator));
      }
      std::cout << '\n';
      i++;
    }
  }
  return 0;
}

Example
One naive implementation is to add both the observed k-mer and its reverse complement as separate k-mers. This is obviously not space efficient but should have fast lookup. This file is called make_struct_lookup.cpp. I used the following command to compile on my Apple laptop (OS X): clang++ -std=c++11 -stdlib=libc++ -Wno-c++98-compat make_struct_lookup.cpp -o msl.
#include <fstream>
#include <string>
#include <map>
#include <iostream>
#include <chrono>
//build the structure. measure how much RAM it consumes.
//then measure how long it takes to lookup in the data structure

#define k 29

std::string rc(std::string seq){
  std::string rc;
  for (int i = seq.length()-1; i>=0; i--){
    if (seq[i] == 'A'){
      rc.push_back('T');
    } else if (seq[i] == 'C'){
      rc.push_back('G');
    } else if (seq[i] == 'G'){
      rc.push_back('C');
    } else if (seq[i] == 'T'){
      rc.push_back('A');
    }
  }
  return rc;
}

int main(int argc, char* argv[]){
  using namespace std::chrono;
  //initialize the data structure
  std::string thisline;
  std::map<std::string, int> kmer_map;
  std::string header;
  std::string seq;
  //open the fasta file
  std::ifstream inFile;
  inFile.open(argv[1]);

  //construct the kmer-lookup structure
  int i = 0;
  high_resolution_clock::time_point t1 = high_resolution_clock::now();
  while (getline(inFile,thisline)){
    if (thisline[0] == '>'){
      header = thisline.substr(1,thisline.size());
      //std::cout << header << '\n';
    } else {
      seq = thisline;
      //now add the kmers
      for (int j=0; j< thisline.size() - k + 1; j++){
        kmer_map[seq.substr(j,j+k)] = stoi(header);
        kmer_map[rc(seq.substr(j,j+k))] = stoi(header);
      }
      i++;
    }
  }
  std::cout << "  -finished " << i << " seqs.\n";
  inFile.close();
  high_resolution_clock::time_point t2 = high_resolution_clock::now();
  duration<double> time_span = duration_cast<duration<double>>(t2 - t1);
  std::cout << time_span.count() << " seconds to load the array." << '\n';

  //now lookup the kmers
  inFile.open(argv[1]);
  t1 = high_resolution_clock::now();
  int lookup;
  while (getline(inFile,thisline)){
    if (thisline[0] != '>'){
      seq = thisline;
      //now lookup the kmers
      for (int j=0; j< thisline.size() - k + 1; j++){
        lookup = kmer_map[seq.substr(j,j+k)];
      }
    }
  }
  std::cout << "  - looked at " << i << " seqs.\n";
  inFile.close();
  t2 = high_resolution_clock::now();
  time_span = duration_cast<duration<double>>(t2 - t1);
  std::cout << time_span.count() << " seconds to lookup the kmers." << '\n';

}

Example output
I ran the above program with the following command to log peak RAM usage. The amount of time the lookup of all k-mers in two million sequences is reported by the program. /usr/bin/time -l ./msl my.fasta.
The output was:
 -finished 2000000 seqs.
562.864 seconds to load the array.
  - looked at 2000000 seqs.
368.734 seconds to lookup the k-mers.
     1046.94 real       942.38 user        78.96 sys
11680514048  maximum resident set size

So, the program used 11680514048 bytes = 11.68GB of RAM and it took 368.734 seconds to lookup the k-mers in two million fasta files.
Results
Below is a plot of the results from each user's answers.

The question and the accepted answer are not about k-mer data structure at all, which I will explain in detail below. I will first answer the actual question OP intends to ask.
The simplest way to keep k-mers is to use an ordinary hash table. The performance is mostly determined by the hash table library. std::unordered_map in gcc/clang is one of the worst choices because for integer keys, it is very slow. Google dense, ska::bytell_hash_map and ska::flat_hash_map, tsl::robin_map and absl::flat_hash_map are much faster. There are a few libraries that focus on smaller footprint, such as google sparse and sparsepp, but those can be a few times slower.
In addition to the choice of hash table, how to construct the key is critical. For k<=32, the right choice is to encode a k-mer with a 64-bit integer, which will be vastly better than std::string. Memory alignment is also important. In C/C++, as long as there is one 8-byte member in a struct, the struct will be 8-byte aligned on x86_64 by default. Most C++ hash table libraries pack key and value in std::pair. If you use 64-bit keys and 32-bit values, std::pair will be 8-byte aligned and use 16 bytes, even though only 12 bytes are actually used – 25% of memory is wasted. In C, we can explicitly define a packed struct with __attribute__ ((__packed__)). In C++, probably you need to define special key types. A better way to get around memory alignment is to go down to the bit level. For read mapping, for example, we only use 15–23bp seeds. Then we have 18 (=64-23*2) bits left unused. We can use these 18 bits to count k-mers. Such bit-level management is quite common.
The above is just basic techniques. There are a few other tricks. For example, 1) instead of using one hash table, we can use 4096 (=2^12) hash tables. Then we can store 12 bits of k-mer information into the 4096 part. This gives us invaluable 12 bits in each bucket to store extra information. This strategy also simplifies parallel k-mer insertions as with a good hash function, it is rare to insert into two tables at the same time. 2) when most k-mers are unique, the faster way to count k-mers is to put k-mers in an array and then sort it. Sorting is more cache friendly and is faster than hash table lookups. The downside is that sort counting can be memory demanding when most k-mers are highly repetitive.

The other answer is spending considerable (probably the majority of) time on k-mer iteration, not on hash table operations. The program loops through each position on the sequence and then each k-mer position. For an $L$-long sequence, this is an $O(kL)$ algorithm. It has worse theoretical time complexity than hash table operations, which is $O(L)$. Although hash table operations are slow due to cache misses, a factor of k=29 is quite significant. Another issue is that all programs in the question and in the other answer are compiled without -O3. Adding this option brings the bytell_hash_map lookup time from 314s to 34s on my machine.
The C program at the end of my post shows the proper way to iterate k-mers. It is an $O(L)$ algorithm with a tiny constant. The program keeps track of both forward and reverse k-mers at the same time and update them with a few bit operations at each sequence position. This echoes my previous comment "You should not reverse complement the whole k-mer". On the same machine, the program looks up k-mers in 5.5s using 792MB RAM at the peak. This 6-fold (=34/5.5) speedup mostly comes from k-mer iteration, given that the hash table library in use is known to have comparable performance to bytell_hash_map.

#include <stdio.h>
#include <stdint.h>
#include "khash.h"

static inline uint64_t hash_64(uint64_t key)
{ // more sophisticated hash function to reduce collisions
  key = (~key + (key << 21)); // key = (key << 21) - key - 1;
  key = key ^ key >> 24;
  key = ((key + (key << 3)) + (key << 8)); // key * 265
  key = key ^ key >> 14;
  key = ((key + (key << 2)) + (key << 4)); // key * 21
  key = key ^ key >> 28;
  key = (key + (key << 31));
  return key;
}

KHASH_INIT(64, khint64_t, int, 1, hash_64, kh_int64_hash_equal)

unsigned char seq_nt4_table[128] = { // Table to change "ACGTN" to 01234
  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,
  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,
  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,
  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,
  4, 0, 4, 1,  4, 4, 4, 2,  4, 4, 4, 4,  4, 4, 4, 4,
  4, 4, 4, 4,  3, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,
  4, 0, 4, 1,  4, 4, 4, 2,  4, 4, 4, 4,  4, 4, 4, 4,
  4, 4, 4, 4,  3, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4
};

static uint64_t process_seq(khash_t(64) *h, int k, int len, char *seq, int is_ins)
{
  int i, l;
  uint64_t x[2], mask = (1ULL<<k*2) - 1, shift = (k - 1) * 2, tot = 0;
  for (i = l = 0, x[0] = x[1] = 0; i < len; ++i) {
    int c = (uint8_t)seq[i] < 128? seq_nt4_table[(uint8_t)seq[i]] : 4;
    if (c < 4) { // not an "N" base
      x[0] = (x[0] << 2 | c) & mask;                  // forward strand
      x[1] = x[1] >> 2 | (uint64_t)(3 - c) << shift;  // reverse strand
      if (++l >= k) { // we find a k-mer
        uint64_t y = x[0] < x[1]? x[0] : x[1];
        khint_t itr;
        if (is_ins) { // insert
          int absent;
          itr = kh_put(64, h, y, &absent);
          if (absent) kh_val(h, itr) = 0;
          tot += ++kh_val(h, itr);
        } else { // look up
          itr = kh_get(64, h, y);
          tot += itr == kh_end(h)? 0 : kh_val(h, k);
        }
      }
    } else l = 0, x[0] = x[1] = 0; // if there is an "N", restart
  }
  return tot;
}

#include <zlib.h>
#include <time.h>
#include <unistd.h>
#include "kseq.h"
KSEQ_INIT(gzFile, gzread)

int main(int argc, char *argv[])
{
  khash_t(64) *h;
  int i, k = 29;
  while ((i = getopt(argc, argv, "k:")) >= 0)
    if (i == 'k') k = atoi(optarg);
  h = kh_init(64);
  for (i = 1; i >= 0; --i) {
    uint64_t tot = 0;
    kseq_t *ks;
    gzFile fp;
    clock_t t;
    fp = gzopen(argv[optind], "r");
    ks = kseq_init(fp);
    t = clock();
    while (kseq_read(ks) >= 0)
      tot += process_seq(h, k, ks->seq.l, ks->seq.s, i);
    fprintf(stderr, "[%d] %.3f\n", i, (double)(clock() - t) / CLOCKS_PER_SEC);
    kseq_destroy(ks);
    gzclose(fp);
  }
  kh_destroy(64, h);
  return 0;
}


Question:
My daughter is in year $3$ and she is now working on subtraction up to $1000.$ She came up with a way of solving her simple sums that we (her parents) and her teachers can't understand.
Here is an example: $61-17$
Instead of borrowing, making it $50+11-17,$ and then doing what she was told in school $11-7=4,$ $50-10=40 \Longrightarrow 40+4=44,$ she does the following:
Units of the subtrahend minus units of the minuend $=7-1=6$
Then tens of the minuend minus tens of the subtrahend $=60-10=50$
Finally she subtracts the first result from the second $=50-6=44$
As it is against the first rule children learn in school regarding subtraction (subtrahend minus minuend, as they cannot invert the numbers in subtraction as they can in addition), how is it possible that this method always works? I have a medical background and am baffled with this…
Could someone explain it to me please? Her teachers are not keen on accepting this way when it comes to marking her exams.

So she is doing
\begin{align*}
61-17=(60+1)-(10+7)&=(60-10)-(7-1)\\
 & = 50-6\\
& =44
\end{align*}
She manage to have positive results on each power of ten group up to a multiplication by $\pm 1$ and sums at the end the pieces ; this is kind of smart :)
Conclusion : If she is comfortable with this system, let her do...


Question:
From school, I remember a very important rule: first you need to pour the water and then the acid (when you need to mix them) not vice-versa. This is because otherwise the aсid becomes very hot and splashing may happen.
So, why does it get hotter when water is poured into it? What reaction takes place?

This is mostly the case for sulfuric acid. Commercially available sulfuric acid is dense (~1.8 g/ml) and when water is added, it may not mix. In this case a layer of hot weak acid solution is formed, which boils and sprays around. When acid is poured into water, it flows down the flask and mixes much better, so no boiling occurs.
The reason this occurs is due to the large amount of energy released in the hydration reaction of sulfuric acid ions. Do not believe that heat comes from dissociation, as the dissociation of acids, bases, and salts always consumes energy. The energy is released from subsequent hydration, and the release may be high, especially if $\ce{H+}$ or $\ce{OH-}$ ions are hydrated.


Question:
The 2018 Nobel Prize in Physics was awarded recently, with half going to Arthur Ashkin for his work on optical tweezers and half going to Gérard Mourou and Donna Strickland for developing a technique called "Chirped Pulse Amplification".
In general, optical tweezers are relatively well known, but Chirped Pulse Amplification is less well understood on a broader physics or optics context. While normally the Wikipedia page is a reasonable place to turn to, in this case it's pretty technical and flat, and not particularly informative. So:

What is Chirped Pulse Amplification? What is the core of the method that really makes it tick?
What pre-existing problems did its introduction solve?
What technologies does it enable, and what research fields have become possible because of it?

The problem
Lasers do all sorts of cool things in research and in applications, and there are many good reasons for it, including their coherence, frequency stability, and controllability, but for some applications, the thing that really matters is raw power.
As a simple example, it had long been understood that if the intensity of light gets high enough, then the assumption of linearity that underpins much of classical optics would break down, and nonlinear optical phenomena like second-harmonic generation would become available, getting light to do all sorts of interesting things. Using incoherent light sources, the required intensities are prohibitively high, but once the laser was invented, it took only one year until the first demonstration of second-harmonic generation, and a few short years after that until third-harmonic generation, a third-order nonlinear process that requires even higher intensities.
Put another way, power matters, and the more intensity you have available, the wider a range of nonlinear optical phenomena will be open for exploration. Because of this, a large fraction of laser science has been focused on increasing the available intensities, generally using pulsed lasers to achieve this and with notable milestones being Q-switching and mode-locking.
However, if you try to push onward with a bigger laser amplifier and more and more power, you are basically destined sooner or later to hit a brick wall, rather brusquely, in the form of catastrophic self-focusing. This is a consequence of yet another nonlinear effect, the Kerr effect, happening inside the laser medium itself. At face value, the Kerr effect looks harmless enough: basically, it says that if the intensity is high enough, the refractive index of the material will rise slightly, in proportion to the intensity:
$$
n(I) = n_0 + n_2\: I.
$$
So, what's the big deal? In short, if you have a laser beam propagating through such a medium, then

the intensity of the light will be higher in the center, which means that the refractive index will be higher in the center.
In other words, the material's optical properties will look like those of a convex lens, and it will tend to focus the beam.
This will tend to make the beam sharper, which will increase the intensity at the center, which will raise the refractive index at the center even higher...
... which will then focus the beam even more tightly, leading to higher and higher intensities.

This makes up a positive feedback loop, and if the initial intensity is high enough, the medium is long enough, and there isn't enough initial diffraction to counteract it, then it will spiral out of control and cause catastrophic laser-induced damage in the very medium that you're trying to use to amplify that laser beam. (Moreover, it is quite common, particularly in air, that the laser will diffract on the damaged spot and then re-self-focus a bit further down the line, a phenomenon known as laser filamentation. If you get things just right wrong, this can propagate a failure in the gain medium up to the destruction of an entire beamline.)

Image source
This sounds like a funky mechanism, but it was a huge roadblock for a very long time. If you plot the highest laser intensity available at different times since the invention of the laser, it climbs quickly up during the sixties, and then it hits a wall and stays put for some ten to fifteen years:

Image source
This represents the barrier of Kerr-lens self-focusing, and at the time the only way to overcome it was to build a laser which was physically bigger, to dilute the intensity over more gain medium to try to prevent the problem. Until, that is, Chirped Pulse Amplification came around to solve the problem.
The solution
At its core, Chirped Pulse Amplification (CPA) works by diluting the light, so that it can be amplified to a larger total power without reaching a dangerous intensity, but it does this stretching in time, i.e. longitudinally along the laser pulse.
The basic sequence consists of four steps:

First of all, you start with a short laser pulse that you want to amplify

You then stretch it in time, by introducing chirp into the signal: that is, you use some sort of dispersive element, like a prism or a diffraction grating, which decomposes the pulse into all of its constituent colors and sends the longer wavelengths first and the shorter wavelengths last. This will naturally reduce the intensity of the pulse.

(Why "chirp"? because the upward (or downward) sweep of frequencies over the pulse is precisely what gives bird chirps their characteristic sound.)
You then pass this lower-intensity pulse through your laser amplifier, which is safe because the instantaneous intensity is below the self-focusing damage threshold of your gain medium.

Finally, you pass your pulse through a reversed set of gratings which will undo the relative delay between the longer- and shorter-wavelengths of your pulse, putting them all together into a single pulse of the same shape and length as your original pulse...

...  but at the much higher amplified power, and at intensities which would be impossible to achieve safely using direct amplification of the pulse.

The core feature that makes the method tick is the fact that, when done correctly, the stretching of the pulse will completely conserve the coherence between the different frequency components, which means that it is fully reversible and when you add a cancelling chirp the pulse will go back to its initial shape.
Furthermore, the method relies on the fact that stimulated emission will completely duplicate, in a coherent way, the photons that it is amplifying, which means that the photons that are introduced by the amplification will have the same frequency and phase characteristics as the initial pulse, which means that when you remove the chirp from the amplified pulse the added-in photons will also compress into a tight envelope.
Applications
Like I said at the beginning, CPA is particularly useful in places where raw laser power, and particularly concentrated laser power, is of paramount importance. Here are some examples:

In the same way that lasers gave us nonlinear optics, CPA has been integral in the development of high-order harmonic generation which has pushed past the second- or third-order harmonics to happily produce tens or hundreds of harmonics. (The current record goes all the way to harmonic 5,000.)
This isn't only 'more', it's qualitatively different: it pushes nonlinear optics to regimes where the usual perturbative expansion completely breaks down, and where it needs to be replaced with a completely new set of tools, which revolve around the so-called three-step model, and which involve a nice and quite particular new interface between classical and quantum mechanics, where trajectories do (sort of) exist but over complex-valued time and space, due to the presence of quantum tunnelling.
It has also helped push the study of light-matter interaction past that same perturbative limit, giving us the tools to extract electrons from molecules and control them in very precise ways, thereby allowing for the creation of tools like e.g. laser-driven electron diffraction, which can be used to image the shapes of molecules as they undergo bending and other vibrations.
CPA also underpins several breakthrough measurements which have been touched on previously on this site, including the observation of the time-dependent waveform of a light pulse, itself done using high-order harmonic radiation; the observation of charge oscillations when atoms are placed in excited states, again using HHG; or performing electron holography from an atomic target using electrons pulled from that same atom.
Of course, all the cool laser-driven QED stuff at the top of that second diagram: if your laser is strong enough that, if you release an electron into the focus, the kinetic energy of its oscillations will exceed $m_e c^2$, then you can start to have things like laser-driven pair creation, and all sorts of fun stuff. Some of it is already on the table, some of it is in achievable plans for the future, and all of it is made possible by CPA.
CPA is also extremely useful in delivering sharply controlled bursts of power to materials. This is extremely useful in laser micromachining, for example, where it is routinely used in e.g. using short laser pulses to etch waveguides into dielectrics, which are then extremely useful for chip-based quantum computation and quantum information processing.
Similarly, the ability to deliver sharply controlled bursts of power is extremely useful in laser microsurgery, and there are several types of eye surgery that exclusively use CPA pulses to provide sharp 'kicks' of power which perform cleaner incisions.
On a much larger scale, when you really turn up the power to the maximum, CPA is a vital component of laser wakefield acceleration, which uses the ionized pocket left behind by an intense laser pulse as it travels through a gas to accelerate electrons to energies that would otherwise require an extremely large particle accelerator, but which are now available using a much more modest table-top laser system.

Further reading
Some additional resources for further reading:

The Nobel Prize's scientific background and popular information documents make excellent reading, and people don't go looking at those sources anywhere near enough. Go check them out!
The original paper: Compression of amplified chirped optical pulses. D. Strickland and G. Mourou. Optics Comms. 55, 447 (1985).
What power limitations was chirped radar designed to overcome?, a previous question of mine on similar radar technologies that predated CPA.
This is a nice tutorial on U Michigan.


Question: I have a DNA sequence for which I would like to quickly find the reverse complement. Is there a quick way of doing this on the bash command line using only GNU tools?

Thanks to Manu Tamminen for this solution:
echo ACCTTGAAA | tr ACGTacgt TGCAtgca | rev


Question:
I am writing a python script that requires a reverse complement function to be called on DNA strings of length 1 through around length 30. Line profiling programs indicate that my functions spend a lot of time getting the reverse complements, so I am looking to optimize.
What is the fastest way to get the reverse complement of a sequence in python? I am posting my skeleton program to test different implementations below with DNA string size 17 as an example.
#!/usr/bin/env python
import random
import timeit

global complement
complement = {'A': 'T', 'C': 'G', 'G': 'C', 'T': 'A'}

DNAlength = 17

#randomly generate 100k bases
int_to_basemap = {1: 'A', 2: 'C', 3: 'G', 4: 'T'}
num_strings = 500000
random.seed(90210)
DNAstrings = ["".join([int_to_basemap[random.randint(1,4)] for i in range(DNAlength)])
              for j in range(num_strings)]
#get an idea of what the DNAstrings look like
print(DNAstrings[0:5])

def reverse_complement_naive(seq):
    this_complement = {'A': 'T', 'C': 'G', 'G': 'C', 'T': 'A'}
    return "".join(this_complement.get(base, base) for base in reversed(seq))

def reverse_complement(seq):
    return "".join(complement.get(base, base) for base in reversed(seq))

tic=timeit.default_timer()
rcs = [reverse_complement_naive(seq) for seq in DNAstrings]
toc=timeit.default_timer()
baseline = toc - tic
namefunc = {"naive implementation": reverse_complement_naive,
            "global dict implementation": reverse_complement}

for function_name in namefunc:
    func = namefunc[function_name]
    tic=timeit.default_timer()
    rcs = [func(seq) for seq in DNAstrings]
    toc=timeit.default_timer()
    walltime = toc-tic
    print("""{}
       {:.5f}s total,
       {:.1f} strings per second
       {:.1f}% increase over baseline""".format(
           function_name,
           walltime,
           num_strings/walltime,
           100- ((walltime/baseline)*100)   ))

By the way, I get output like this. It varies by the call, of course!
naive implementation
       1.83880s total,
       271916.7 strings per second
       -0.7% increase over baseline
global dict implementation
       1.74645s total,
       286294.3 strings per second
       4.3% increase over baseline

Edit: Great answers, everyone! When I get a chance in a day or two I will add all of these to a test file for the final run. When I asked the question, I had not considered whether I would allow for cython or c extensions when selecting the final answer. What do you all think?
Edit 2: Here are the results of the final simulation with everyone's implementations. I am going to accept the highest scoring pure python code with no Cython/C. For my own sake I ended up using user172818's c implementation. If you feel like contributing to this in the future, check out the github page I made for this question.
the runtime of reverse complement implementations.
10000 strings and 250 repetitions
╔══════════════════════════════════════════════════════╗
║ name                       %inc   s total  str per s ║
╠══════════════════════════════════════════════════════╣
║ user172818 seqpy.c        93.7%  0.002344  4266961.4 ║
║ alexreynolds Cython (v2)  93.4%  0.002468  4051583.1 ║
║ alexreynolds Cython (v1)  90.4%  0.003596  2780512.1 ║
║ devonryan string          86.1%  0.005204  1921515.6 ║
║ jackaidley bytes          84.7%  0.005716  1749622.2 ║
║ jackaidley bytesstring    83.0%  0.006352  1574240.6 ║
║ global dict                5.4%  0.035330   283046.7 ║
║ revcomp_translateSO       45.9%  0.020202   494999.4 ║
║ string_replace            37.5%  0.023345   428364.9 ║
║ revcom from SO            28.0%  0.026904   371694.5 ║
║ naive (baseline)           1.5%  0.036804   271711.5 ║
║ lambda from SO           -39.9%  0.052246   191401.3 ║
║ biopython seq then rc    -32.0%  0.049293   202869.7 ║
╚══════════════════════════════════════════════════════╝

I don't know if it's the fastest, but the following provides an approximately 10x speed up over your functions:
import string
tab = string.maketrans("ACTG", "TGAC")

def reverse_complement_table(seq):
    return seq.translate(tab)[::-1]

The thing with hashing is that it adds a good bit of overhead for a replacement set this small.
For what it's worth, I added that to your code as "with a translation table" and here is what I got on my workstation:
global dict implementation
       1.37599s total,
       363374.8 strings per second
       3.3% increase over baseline
naive implementation
       1.44126s total,
       346919.4 strings per second
       -1.3% increase over baseline
with a translation table
       0.16780s total,
       2979755.6 strings per second
       88.2% increase over baseline

If you need python 3 rather than python 2, then substitute tab = str.maketrans("ACTG", "TGAC") for tab = string.maketrans("ACTG", "TGAC"), since maketrans is now a static method on the str type.
For those wondering, using biopython is slower for this (~50% slower than the naive implementation), presumably due to the overhead of converting the strings to Seq objects. If one were already reading sequences in using biopython, though, I wouldn't be surprised if the performance was much different.


Question:
Below is a signal which represents a recording of someone talking. I would like to create a series of smaller audio signals based on this. The idea being to detect when 'important' sound starts and ends and use those for markers to make new snippet of audio. In other words, I would like to use the silence as indicators as to when an audio 'chunk' has started or stopped and make new audio buffers based on this.
So for example, if a person records himself saying
Hi [some silence] My name is Bob [some silence] How are you?

then I would like to make three audio clips from this. One that says Hi, one that says My name is Bob and one that says How are you?.
My initial idea is to run through the audio buffer constantly checking where there are areas of low amplitude. Maybe I could do this by taking the first ten samples, average the values and if the result is low then label it as silent. I would proceed down the buffer by checking the next ten samples. Incrementing  along in this way I could detect where envelopes start and stop.
If anyone has any advice on a good, but simple way to do this that would be great. For my purposes the solution can be quite rudimentary.
I'm not a pro at DSP, but understand some basic concepts. Also, I would be doing this programmatically so it would be best to talk about algorithms and digital samples.
Thanks for all the help!

EDIT 1
Great responses so far! Just wanted to clarify that this is not on live audio and I will be writing the algorithms myself in C or Objective-C so any solutions that use libraries aren't really an option.

What you really want to do is essentially called as Voice Activity Detection or speech detection.
Basically any pure speech signal (which contains no music) has three parts.

The voiced sound - which is basically caused by Vowels
The unvoiced sound - which contains consonants.

The characteristic of human sound is such that while a lot of energy is used in voiced sound the real information is contained in consonants. Also, voiced sound is usually lower frequency where as unvoiced sounds are higher frequencies. [To be precise all voiced sound are resonated more or less a constant frequency for a given person which is his/her pitch].
Now, as any system there is noise. The voiced sound is usually quite powerful enough that it can be distinguished visible. When you apply a lower frequency filtering it is possible to collect good magnitude of voiced sounds however, the unvoiced sound (with all the rich information) will get lost.
Coming to the question how to solve it:
The trick lies in the fact that unvoiced sound still come from a resonating source; and inherently restricted over a certain frequency. Where as, the noise is rather uniform. So a simple measure that distinguish all three is "local power" or alternatively but equivalent is to take the windowed auto-correlation.
If you take at a time say 100 samples - and auto correlate itself, if it contains only noise the results will be pretty much zero (this is the property of white noise) where as for speech signal, this magnitude will be observable because the signal still has better structure.  This has worked for me in the past.
VAD has been an active research areas- because almost all Mobile phone communications wants to detect non speech part and remove them from encoding. But if they would remove non-voiced speech this would make telephony useless.
The G.729 standard computes VAD based on features like: line spectral frequencies, full-band energy, low-band energy (<1 kHz), and zero-crossing rate.
The GSM standard works as follows: Option 1 computes the SNR in nine bands and applies a threshold to these values. Option 2 calculates different parameters: channel power, voice metrics, and noise power. It then thresholds the voice metrics using a threshold that varies according to the estimated SNR. (from wikipedia)
For more advanced techniques i am listing some references on this subject.

Most sited reference: Jongseo Sohn;   Nam Soo Kim;   Wonyong Sung;  "A statistical model-based voice activity detection" Signal Processing Letters, IEEE, Jan 1999, Volume: 6 Issue:1 pp:1-3
Most relevant for you: Mark Marzinzik and Birger Kollmeier "Speech Pause Detection for Noise Spectrum Estimation by Tracking Power Envelope Dynamics" IEEE TRANSACTIONS ON SPEECH AND AUDIO PROCESSING, VOL. 10, NO. 2, FEBRUARY 2002 pp.109
Ramírez, J.; J. M. Górriz, J. C. Segura (2007). "Voice Activity Detection. Fundamentals and Speech Recognition System Robustness". In M. Grimm and K. Kroschel. Robust Speech Recognition and Understanding. pp. 1–22. ISBN 978-3-902613-08-0.
Introductory : Jonathan Kola, Carol Espy-Wilson and Tarun Pruthi "Voice Activity Detection"


Question:
It seems that a well-designed SMPS has a capacitor connecting the ground planes of the primary and secondary sides of the transformer, such as the C13 capacitor here. What is the purpose of this capacitor?
I've let myself understand that it's for EMI suppression, but what kind of EMI does it suppress, and how? It seems to me to be the only leg of an open circuit and thus completely inert, but obviously I'm wrong about that.

Switched mode power supplies use what is known as a "flyback converter" to provide voltage conversion and galvanic isolation. A core component of this converter is a high frequency transformer.
Practical transformers have some stray capacitance between primary and secondary windings. This capacitance interacts with the switching operation of the converter. If there is no other connection between input and output this will result in a high frequency voltage between the output and input.
This is really bad from an EMC perspective. The cables from the power brick are now essentially acting as an antenna transmitting the high frequency generated by the switching process.
To suppress the high frequency common mode is is necessary to put capacitors between the input and output side of the power supply with a capacitance substantially higher than the capacitance in the flyback transformer. This effectively shorts out the high frequency and prevents it escaping from the device.
When desinging a class 2 (unearthed) PSU we have no choice but to connect these capacitors to the input "live" and/or "neutral". Since most of the world doesn't enforce polarity on unearthed sockets we have to assume that either or both of the "live" and "neutral" terminals may be at a sinificant voltage relative to earth and we usually end up with a symmetrical design as a "least bad option". That is why if you measure the output of a class 2 PSU relative to mains earth with a high impedance meter you will usually see around half the mains voltage.
That means on a class 2 PSU we have a difficult tradeoff between safety and EMC. Making the capacitors bigger improves EMC but also results in higher "touch current" (the current that will flow through someone or something who touches the output of the PSU and mains earth). This tradeoff becomes more problematic as the PSU gets bigger (and hence the stray capacitance in the transformer gets bigger).
On a class 1 (earthed) PSU we can use the mains earth as a barrier between input and output either by connecting the output to mains earth (as is common in desktop PC PSUs) or by using two capacitors, one from the output to mains earth and one from mains earth to the input (this is what most laptop power bricks do). This avoids the touch current problem while still providing a high frequency path to control EMC.
Short circuit failure of these capacitors would be very bad. In a class 1 PSU failure of the capacitor between the mains supply and mains earth would mean a short to earth, (equivalent to a failure of "basic" insulation). This is bad but if the earthing system is functional it shouldn't be a major direct hazard to users. In a class 2 PSU a failure of the capacitor is much worse, it would mean a direct and serious safety hazard to the user (equivilent to a failure or "double" or "reinforced" insulation). To prevent hazards to the user the capacitors must be designed so that short circuit failure is very unlikely.
So special capacitors are used for this purpose. These capacitors are known as "Y capacitors" (X capacitors on the other hand are used between mains live and mains neutral). There are two main subtypes of "Y capacitor", "Y1" and "Y2" (with Y1 being the higher rated type). In general Y1 capacitors are used in class 2 equipment while Y2 capacitors are used in class 1 equipment.

So does that capacitor between the primary and secondary sides of the SMPS mean that the output is not isolated? I've seen lab supplies that can be connected in series to make double the voltage. How do they do that if it isn't isolated?

Some power supplies have their outputs hard-connected to earth. Obviously you can't take a pair of power supplies that have the same output terminal hard-connected to earth and put them in series.
Other power supplies only have capactive coupling from the output to either the input or to mains earth. These can be connected in series since capacitors block DC (though don't go crazy, the capacitors will have a limited working voltage).


Question:
Move a match slowly and nothing happens but if you shake it violently the fire will extinguish. Oxygen makes fire grow so why does waving a flame through the oxygen-rich air put the fire out? Does this primarily have to do with a decrease in the temperature of the burning materials or is it something else?
Also what about forest fires, do high winds spread or kill the fire?

Combustion of small materials, such as a match or birthday candle, actually involves the release of volatile vapours, which themselves burn. It is not the solid material that burns. There needs to be a minimum amount of volatile material present in this combustion zone (just above the burning match) for the ignition to occur. As the combustion process continues, heat is given off, and more volatile materials are released, which in turn continues the combustion cycle. Now, if you shake a match or blow on a candle, you rapidly disperse these volatile fuels from the combustion zone, and there is no longer sufficient fuel to ignite. It is effectively removing the fuel component of the fire triangle, for just a brief moment.
Large fires can reignite because there is sufficient heat left in the fuel to further release volatile fuels, which can either self-ignite or ignite through the presence of embers.
Blowing gently on a small wood fire increases the oxygen content in the combustion zone, without dispersing the fuel. Similarly, if you experiment with a small enough wood fire, you will see that blowing on different parts of the fire will have a different outcome: blowing on the base of the fuels will increase oxygen content, and not affect volatile fuels. Blowing on the top of the fire (where the flame starts to burn) will probably put the fire out.
Will this put out a small paper fire? That will depend on the heat retained by the burning fuel. A single piece of A4 paper if shaken hard enough will extinguish. A ream of A4 paper that has burned halfway down the page will be put out this way, but could easily reignite if the paper pages are pulled apart to allow oxygen into the released volatile fuels.
Generally, forest fires are accelerated by strong winds. Winds affect forest fires in a number of ways and increase the rate of spread significantly. This is a topic for a whole other question.


Question:
People often say that LR(k) parsers are more powerful than LL(k) parsers. These statements are vague most of the time; in particular, should we compare the classes for a fixed $k$ or the union over all $k$? So how is the situation really? In particular, I am interested in how LL(*) fits in.
As far as I know, the respective sets of grammars LL and LR parsers accept are orthogonal, so let us talk about the languages generated by the respective sets of grammars. Let $LR(k)$ denote the class of languages generated by grammars that can be parsed by an $LR(k)$ parser, and similar for other classes.
I am interested in the following relations:

$LL(k) \overset{?}{\subseteq} LR(k)$
$\bigcup_{i=1}^{\infty} LL(k) \overset{?}{\subseteq} \bigcup_{i=1}^{\infty} LR(k)$
$\bigcup_{i=1}^{\infty} LL(k) \overset{?}{=} LL(*)$
$LL(*) \overset{?}{\circ} \bigcup_{i=1}^{\infty} LR(k)$

Some of these are probably easy; my goal is to collect a "complete" comparison. References are appreciated.

There are numerous containments known. Let $\subseteq$ denote containment and $\subset$ proper containment. Let $\times$ denote incomparability.
Let $LL = \bigcup_k LL(k)$, $LR = \bigcup_k LR(k)$.
Grammar level
For LL

$LL(0) \subset LL(1) \subset LL(2) \subset LL(2) \subset \cdots \subset LL(k) \subset \cdots \subset LL \subset LL(*)$
$SLL(1) = LL(1), SLL(k) \subset LL(k), SLL(k+1) \times LL(k)$

Most of these are proven in Properties of deterministic top down grammars by Rosenkrantz and Stearns. $SLL(k+1) \times LL(k)$ is a rather trivial exercise. This presentation by Terence Parr places $LL(*)$ on slide 13. The paper LL-regular grammars by Jarzabek and Krawczyk show $LL \subset LLR$, and their proof trivially extends to $LL \subset LL(*)$
For LR

$LR(0) \subset SLR(1) \subset LALR(1) \subset LR(1)$
$SLR(k) \subset LALR(k) \subset LR(k)$
$SLR(1) \subset SLR(2) \subset \cdots \subset SLR(k)$
$LALR(1) \subset LALR(2) \subset \cdots \subset LALR(k)$
$LR(0) \subset LR(1) \subset LR(2) \subset \cdots \subset LR(k) \subset \cdots \subset LR$

These are all simple exercises.
LL versus LR

$LL(k) \subset LR(k)$ (Properties of deterministic top down grammars plus any left recursive grammar)
$LL(k) \times SLR(k), LALR(k), LR(k-1)$ (simple exercise)
$LL \subset LR$ (any left recursive grammar)
$LL(*) \times LR$ (left recursion versus arbitrary lookahead)

Language level
For LL

$LL(0) \subset LL(1) \subset LL(2) \subset \cdots \subset LL(k) \subset \cdots \subset LL \subset LL(*)$
$SLL(k) = LL(k)$

Most of these are proven in Properties of deterministic top down grammars. The equivalence problem for LL- and LR-regular grammars by Nijholt makes references to papers showing $LL(k) \subset LL(*)$. The paper LL-regular grammars by Jarzabek and Krawczyk show $LL \subset LLR$, and their proof trivially extends to $LL \subset LL(*)$
For LR

$LR(0) \subset SLR(1) = LALR(1) = LR(1) = SLR(k) = LALR(k) = LR(k) = LR$

Some of these were proven by Knuth in his paper On the Translation of Languages from Left to Right in which he introduced LR(k), the rest are proven in Transforming LR(k) Grammars to LR(1), SLR(1), and (1,1) Bounded Right-Context Grammars by Mickunas et al.
LL versus LR

$LL \subset LR(1)$ (containment follows from the above, $\{ a^i b^j | i \geq j \}$ is the canonical example for strict containment)
$LL(*) \times LR$ (the language $\{ a^i b^j | i \geq j \}$ shows half the claim, and the introduction of The equivalence problem for LL- and LR-regular grammars by Nijholt makes references to papers showing the other half)
$LR(1) = DCFL$ (see e.g. reference here).


Question:
In this YouTube video from Cody's Lab, Cody claims that heavy water tastes sweet.
He does some fairly convincing comparisons but still expresses a little doubt that the effect is real.
Has this been studied by others? Is the effect verified and if so, what possible explanations exist?

According to H.C. Urey and G. Failla, Science  15 Mar 1935,
Vol. 81, Issue 2098, pp. 273, there's no difference in the taste of ordinary and heavy water.


Question:
When designing a digital filter based on an analog filter we usually use the bilinear transform. To approximate a discrete transfer function $D_a(z)$ from analog (continuous) transfer function $A(s)$ we substitute
$$z = \frac{1+sT/2}{1-sT/2}$$
where $T$ is the sampling period. Alternatively, to approximate an continuous transfer function $A_a(s)$ from discrete transfer function $D(z)$ we substitute
$$s = \frac{2}{T} \frac{z-1}{z+1}$$
Are there alternative methods of performing such conversions? Are there better approximations?

Analog filters are stable if the poles are in the left half of the s-plane (figure on the left) and digital filters are stable if the poles are inside the unit circle (figure on the right). So mathematically all that is needed to convert from analog to digital is a mapping (conformal?) from the half-space to the unit disk and the $\jmath\Omega$ axis to the unit circle $\vert z\vert=1$. Any transformation that does this is a possible candidate for being an alternative to the bilateral transform.

Two of the well known methods are the impulse invariance method and the matched Z-transform method. Conceptually, both of these are similar to sampling a continuous waveform that we're familiar with. Denoting the inverse Laplace transform by $\mathcal{L}^{-1}$ and the Z transform as $\mathcal{Z}$, both these methods involve calculating the impulse response of the analog filter as
$$a(t)=\mathcal{L}^{-1}\{A(s)\}$$
and sampling $a(t)$ at a sampling interval $T$ that is high enough so as to avoid aliasing. The transfer function of the digital filter is then obtained from the sampled sequence $a[n]$ as
$$D_a(z)=\mathcal{Z}\{a[n]\}$$
However, there are key differences between the two.
Impulse invariance method:
In this method, you expand the analog transfer function as partial fractions (not in the matched Z transform as mentioned by Peter) as
$$A(s)=\sum_m \frac{C_m}{s-\alpha_m}$$
where $C_m$ is some constant and $\alpha_m$ are the poles. Mathematically, any transfer function with a numerator of lesser degree than the denominator can be expressed as a sum of partial fractions. Only low-pass filters satisfy this criterion (high-pass and bandpass/bandstop have at least the same degree), and hence impulse invariant method cannot be used to design other filters.
The reason why it fails is also quite clear. If you had a polynomial in the numerator of the same degree as in the denominator, you will have a free standing constant term, which upon inverse transforming, will give a delta function that cannot be sampled.
If you carry out the inverse Laplace and forward Z transforms, you'll see that the poles are transformed as $\alpha_m \to e^{\alpha_m T}$ which means that if your analog filter is stable, the digital filter will also be stable. Hence it preserves the stability of the filter.
Matched Z-transform
In this method, instead of splitting the impulse response as partial fractions, you do a simple transform of both the poles and the zeros in a similar manner (matched) as $\beta_m\to e^{\beta_m T}$ and $\alpha_m\to e^{\alpha_m T}$ (also stability preserving), giving
$$A(s)=\frac{\prod_m (s-\beta_m)}{\prod_n (s-\alpha_n)}\longrightarrow \frac{\prod_m \left(1-z^{-1}e^{\beta_m T}\right)}{\prod_n \left(1-z^{-1}e^{\alpha_n T}\right)}$$
You can easily see the limitation of both these methods. Impulse invariant is applicable only if your filter is low pass and matched z-transform method is applicable to bandstop and bandpass filters (and high pass up to the Nyquist frequency). They are also limited in practice by the sampling rate (after all, you can only go up to a certain point) and suffer from the effects of aliasing.
The bilinear transform is by far the most commonly used method in practice and the above two are rather more for academic interests. As for conversion back to analog, I'm sorry but I do not know and can't be of much help there as I hardly ever use analog filters.


Question:
I have browsed several ASIC manufacturer's webs, but I haven't found an actual number.
I assume there would be a fixed cost associated with creating masks and such and then there will be a cost per unit.

Note: that I don't actually want to have an ASIC made, I'm just curious.

I looked into ASIC's a while ago and here's what I found:
Everybody has different definitions for the word "ASIC".  There are (very roughly) three categories:  FPGA Conversions, "normal" ASIC, and "full custom".  As expected, these are in order of increasing price and increasing performance.
Before describing what these are, let me tell you how a chip is made...  A chip has anywhere from 4 to 12+ "layers".  The bottom 3 or 4 layers contains the transistors and some basic interconnectivity.  The upper layers are almost entirely used to connect things together.  "Masks" are kind-of like the transparencies used in the photo-etching of a PCB, but there is one mask per IC layer.
When it comes to making an ASIC, the cost of the masks is HUGE.  It is not uncommon at all for a set of masks (8 layers, 35 to 50 nm) to run US$1 Million!  So it is no great surprise to know that most of the "cheaper" ASIC suppliers try very hard to keep the costs of the masks down.
FPGA Conversions:  There are companies that specialize in FPGA to ASIC conversions.  What they do is have a somewhat standard or fixed "base" which is then customized.  Essentially the first 4 or 5 layers of their chip is the same for all of their customers.  It contains some logic that is similar to common FPGA's.  Your "customized" version will have some additional layers on top of it for routing.  Essentially you're using their logic, but connecting it up in a way that works for you.  Performance of these chips is maybe 30% faster than the FPGA you started with. Back in "the day", this would also be called a "sea of gates" or "gate array" chip.
Pros:  Low NRE (US$35k is about the lowest).  Low minimum quantities (10k units/year).
Cons:  High per-chip costs-- maybe 50% the cost of an FPGA.  Low performance, relative to the other solutions.
"Normal" ASIC:  In this solution, you are designing things down to the gate level.  You take your VHDL/Verilog and compile it.  The design for the individual gates are taken from a library of gates & devices that has been approved by the chip manufacturer (so they know it works with their process).  You pay for all the masks, etc.
Pros:  This is what most of the chips in the world are.  Performance can be very good.  Per-chip costs is low.
Cons:  NRE for this starts at US$0.5 million and quickly goes up from there.  Design verification is super important, since a simple screw-up will cost a lot of money.  NRE+Minimum order qty is usually around US$1 million.
Full Custom:  This is similar to a Normal ASIC, except that you have the flexibility to design down to the transistor level (or below).  If you need to do analog design, super low power, super high performance, or anything that can't be done in a Normal ASIC, then this is the thing for you.
Pros:  This requires a very specialized set of talents to do properly.  Performance is great.
Cons:  Same con's as Normal ASIC, only more so.  Odds of screwing something up is much higher.
How you go about this really depends on how much of the work you want to take on.  It could be as "simple" as giving the design files to a company like TSMC or UMC and they give you back the bare wafers.  Then you have to test them, cut them apart, package them, probably re-test, and finally label them.  Of course there are other companies that will do most of that work for you, so all you get back are the tested chips ready to be put on a PCB.
If you have gotten to this point and it still seems like an ASIC is what you want to do then the next step would be to start Googling for companies and talking with them.  All of those companies are slightly different, so it makes sense to talk with as many of them as you can put up with.  They should also be able to tell you what the next step is beyond talking with them.


Question:
Welch's method has been my go-to algorithm for computing power spectral density (PSD) of evenly-sampled timeseries.  I noticed that there are many other methods for computing PSD.  For example, in Matlab I see:

PSD using Burg method
PSD using covariance method
PSD using periodogram
PSD using modified covariance method
PSD using multitaper method (MTM)
PSD using Welch's method
PSD using Yule-Walker AR method
Spectrogram using short-time Fourier transform
Spectral estimation

What are the advantages of these various methods?  As a practical question, when would I want to use something other than Welch's method?

I have no familiarity with the Multitaper method. That said, you've asked quite a question. In pursuit of my MSEE degree, I took an entire course that covered PSD estimation. The course covered all of what you listed (with exception to the Multitaper method), and also subspace methods. Even this only covers some of the main ideas, and there are many methods stemming from these concepts.
For starters, there are two main methods of power spectral density estimation: non-parametric and parametric.
Non-parametric methods are used when little is known about the signal ahead of time. They typically have less computational complexity than parametric models. Methods in this group are further divided into two categories: periodograms and correlograms. Periodograms are also sometimes referred to as direct methods, as they result in a direct transformation of the data. These include the sample spectrum, Bartlett's method, Welch's method, and the Daniell Periodogram. Correlograms are sometimes referred to as indirect methods, as they exploit the Wiener-Khinchin theorem. Therefore these methods are based on taking the Fourier transform of some sort of estimate of the autocorrelation sequence. Because of the high amount of variance associated with higher order lags (due to a small amount of data samples used in the correlations), windowing is used. The Blackman-Tukey method generalizes the correlogram methods.
Parametric methods typically assume some sort of signal model prior to calculation of the PSD estimate. Therefore, it is assumed that some knowledge of the signal is known ahead of time. There are two main parametric method categories: autoregressive methods and subspace methods.
Autoregressive methods assume that the signal can be modeled as the output of an autoregressive filter (such as an IIR filter) driven by a white noise sequence. Therefore all of these methods attempt to solve for the IIR coefficients, whereby the resulting power spectral density is easily calculated. The model order (or number of taps), however, must be determined. If the model order is too small, the spectrum will be highly smoothed, and lack resolution. If the model order is too high, false peaks from an abundant amount of poles begin to appear. If the signal may be modeled by an AR process of model 'p', then the output of the filter of order >= p driven by the signal will produce white noise. There are hundreds of metrics for model order selection. Note that these methods are excellent for high-to-moderate SNR, narrowband signals. The former is because the model breaks down in significant noise, and is better modeled as an ARMA process. The latter is due to the impulsive nature of the resulting spectrum from the poles in the Fourier transform of the resulting model. AR methods are based on linear prediction, which is what's used to extrapolate the signal outside of its known values. As a result, they do not suffer from sidelobes and require no windowing.
Subspace methods decompose the signal into a signal subspace and noise subspace. Exploiting orthogonality between the two subspaces allows a pseudospectrum to be formed where large peaks at narrowband components can appear. These methods work very well in low SNR environments, but are computationally very expensive. They can be grouped into two categories: noise subspace methods and signal subspace methods.
Both categories can be utilized in one of two ways: eigenvalue decomposition of the autocorrelation matrix or singular value decomposition of the data matrix.
Noise subspace methods attempt to solve for 1 or more of the noise subspace eigenvectors. Then, the orthogonality between the noise subspace and the signal subspace produces zeros in the denominator of the resulting spectrum estimates, resulting in large values or spikes at true signal components. The number of discrete sinusoids, or the rank of the signal subspace, must be determined/estimated, or known ahead of time.
Signal subspace methods attempt to discard the noise subspace prior to spectral estimation, improving the SNR. A reduced rank autocorrelation matrix is formed with only the eigenvectors determined to belong to the signal subspace (again, a model order problem), and the reduced rank matrix is used in any one of the other methods.
Now, I'll try to quickly cover your list:

PSD using Burg method: The Burg method leverages the Levinson recursion slightly differently than the Yule-Walker method, in that it estimates the reflection coefficients by minimizing the average of the forward and backward linear prediction error. This results in a harmonic mean of the partial correlation coefficients of the forward and backward linear prediction error. It produces very high resolution estimates, like all autoregressive methods, because it uses linear prediction to extrapolate the signal outside of its known data record. This effectively removes all sidelobe phenomena. It is superior to the YW method for short data records, and also removes the tradeoff between utilizing the biased and unbiased autocorrelation estimates, as the weighting factors divide out. One disadvantage is that it can exhibit spectral line splitting. In addition, it suffers from the same problems all AR methods have. That is, low to moderate SNR severely degrades the performance, as it is no longer properly modeled by an AR process, but rather an ARMA process. ARMA methods are rarely used as they generally result in a nonlinear set of equations with respect to the moving average parameters.

PSD using covariance method: The covariance method is a special case of the least-squares method, whereby the windowed portion of the linear prediction errors is discarded. This has superior performance to the Burg method, but unlike the YW method, the matrix inverse to be solved for is not Hermitian Toeplitz in general, but rather the product of two Toeplitz matrices. Therefore, the Levinson recursion cannot be used to solve for the coefficients. In addition, the filter generated by this method is not guaranteed to be stable. However, for spectral estimation this is a good thing, resulting in very large peaks for sinusoidal content.

PSD using periodogram: This is one of the worst estimators, and is a special case of Welch's method with a single segment, rectangular or triangular windowing (depending on which autocorrelation estimate is used, biased or unbiased), and no overlap. However, it's one of the "cheapest" computationally speaking. The resulting variance can be quite high.

PSD using modified covariance method: This improves on both the covariance method and the Burg method. It can be compared to the Burg method, whereby the Burg method only minimizes the average forward/backward linear prediction error with respect to the reflection coefficient, the MC method minimizes it with respect to ALL of the AR coefficients. In addition, it does not suffer from spectral line splitting, and provides much less distortion than the previously listed methods. In addition, while it does not guarantee a stable IIR filter, it's lattice filter realization is stable. It is more computationally demanding than the other two methods as well.

PSD using Welch's method: Welch's method improves upon the periodogram by addressing the lack of the ensemble averaging which is present in the true PSD formula. It generalizes Barlett's method by using overlap and windowing to provide more PSD "samples" for the pseudo-ensemble average. It can be a cheap, effective method depending on the application. However, if you have a situation with closely spaced sinusoids, AR methods may be better suited. However, it does not require estimating the model order like AR methods, so if little is known about your spectrum a priori, it can be an excellent starting point.

PSD using Yule-Walker AR method: This is a special case of the least squares method where the complete error residuals are utilized. This results in diminished performance compared to the covariance methods, but may be efficiently solved using the Levinson recursion. It's also known as the autocorrelation method.

Spectrogram using short-time Fourier transform: Now you're crossing into a different domain. This is used for time-varying spectra. That is, one whose spectrum changes with time. This opens up a whole other can of worms, and there are just as many methods as you have listed for time-frequency analysis. This is certainly the cheapest, which is why its so frequently used.

Spectral estimation: This is not a method, but a blanket term for the rest of your post. Sometimes the Periodogram is referred to as the "sample spectrum" or the "Schuster Periodogram", the former of which may be what you're referring to.

If you are interested, you may also look into subspace methods such as MUSIC and Pisarenko Harmonic Decomposition. These decompose the signal into signal and noise subspace, and exploits the orthogonality between the noise subspace and the signal subspace eigenvectors to produce a pseudospectrum. Much like the AR methods, you may not get a "true" PSD estimate, in that power most likely is not conserved, and the amplitudes between spectral components is relative. However, it all depends on your application.


Question:
First, sorry if I am missing something basic - I am a programmer recently turned bioinformatician so I still don't know a lot of stuff. This is a cross post with a Biostars question hope that's not bad form.

While it is obvious that scRNA-seq data contain lots of zeroes, I couldn't find any detailed explanation of why they occur except for short notices along the lines of "substantial technical and biological noise". For the following text, let's assume we are looking at a single gene that is not differentially expressed across cells.
If zeroes were caused solely by low capture efficiency and sequencing depth, all observed zeroes should be explained by low mean expression across cells. This however does not seem to be the case as the distribution of gene counts across cells often has more zeroes than would be expected from a negative binomial model. For Example the ZIFA paper explicitly uses a zero-inflated negative binomial distribution to model scRNA-seq data. Modelling scRNA-seq as zero-inflated negative binomial seems widespread throughout the literature.
However assuming negative binomial distribution for the original counts (as measured in bulk RNA-seq) and assuming that every RNA fragment of the same gene from every cell has approximately the same (low) chance of being captured and sequenced, the distribution across single cells should still be negative binomial (see this question  for related math).
So the only remaining possible cause is that inflated zero counts are caused by PCR. Only non-zero counts (after capture) are amplified and then sequenced, shifting the mean of the observed gene counts away from zero while the pre-PCR zero counts stay zero. Indeed some quick simulations show that such a procedure could occasionally generate zero-inflated negative binomial distributions. This would suggest that excessive zeroes should not be present when UMIs are used - I checked one scRNA-seq dataset with UMIs and it seems to be fit well by plain negative binomial.
Is my reasoning correct? Thanks for any pointers.
The question
How can we distinguish between true zero and dropout-zero counts in single-cell RNA-seq? is related, but provides no clues to my present inquiry.

It may be necessary to distinguish between methods that use unique molecular identifiers (UMIs), such as 10X's Chromium, Drop-seq, etc, and non-UMI methods, such as SMRT-seq.  At least for UMI-based methods, the alternative perspective, that there is no significant zero-inflation in scRNA-seq, is also advocated in the single-cell research community.  The argument is straight-forward: the empirical mean expression vs. dropout rate curve matches the theoretically predicted one, given the current levels of capture efficiency.
Examples
Svensson Blog
A couple of blog posts from Valentine Svensson argue this point rather pedagogically, and include citations from across the literature:
Droplet scRNA-seq is not zero-inflated
Count-depth variation makes Poisson scRNA-seq data negative binomial
bayNorm
There is a more extensive preprint by Tang, Shahrezaei, et al. (BioRxiv, 2018) that claims to show a binomial model is sufficient to account for the observed dropout noise.  Here is a snippet of a relevant conclusion:

Importantly, as bayNorm recovered dropout rates successfully in both UMI-based and non-UMI protocols without the need for specific assumptions, we conclude that invoking zero-inflation models is not required to describe scRNA-seq data.  Consistent with this, the differences in mean expression levels of lowly expressed genes observed between bulk and scRNA-seq data, which were suggested to be indicative of zero-inflation, were recovered by our simulated data using the binomial model only.

Multinomial Modeling
There is also a very clearly written preprint by Townes, Irizarry, et al. (BioRxiv, 2019) where the authors consider scRNA-seq as a proper compositional sampling (i.e., multinomial process) and they come to a similar conclusion, though specifically for UMI-based methods. From the paper:

The multinomial model makes two predictions which we verified using negative control data. First, the fraction of zeros in a sample (cell or droplet) is inversely related to the total number of UMIs in that sample. Second, the probability of an endogenous gene or ERCC spike-in having zero counts is a decreasing function of its mean expression (equations provided in Methods). Both of these predictions were validated by the negative control data (Figure 1). In particular, the empirical probability of a gene being zero across droplets was well calibrated to the theoretical prediction based on the multinomial model.  This also demonstrates that UMI counts are not zero inflated.

Furthermore, by comparing raw read counts (prior to UMI-based deduplication) and UMI counts, they conclude that PCR is indeed the cause of zero-inflation:

The results suggest that while read counts appear zero-inflated and multimodal, UMI counts follow a discrete distribution with no zero inflation (Figure S1). The apparent zero inflation in read counts is a result of PCR duplicates.

I highly recommend giving this a read, especially because it nicely situates other common generative models (e.g., binomial, Poisson) as valid simplifying assumptions of the multinomial model.
It should be noted that this same group previously published a work (Hicks, Irizarry, et al. 2018), mostly focused on non-UMI-based datasets (SMRT-seq), where they showed evidence that, relative to bulk RNA-seq, there was significant zero-inflation.


Question:
The Vapnik–Chervonenkis (VC)-dimension formula for neural networks ranges from $O(E)$ to $O(E^2)$, with $O(E^2V^2)$ in the worst case, where $E$ is the number of edges and $V$ is the number of nodes. The number of training samples needed to have a strong guarantee of generalization is linear with the VC-dimension.
This means that for a network with billions of edges, as in the case of successful deep learning models, the training dataset needs billions of training samples in the best case, to quadrillions in the worst case. The largest training sets currently have about a hundred billion samples.  Since there is not enough training data, it is unlikely deep learning models are generalizing. Instead, they are overfitting the training data.  This means the models will not perform well on data that is dissimilar to the training data, which is an undesirable property for machine learning.
Given the inability of deep learning to generalize, according to VC dimensional analysis, why are deep learning results so hyped? Merely having a high accuracy on some dataset does not mean much in itself. Is there something special about deep learning architectures that reduces the VC-dimension significantly?
If you do not think the VC-dimension analysis is relevant, please provide evidence/explanation that deep learning is generalizing and is not overfitting. I.e. does it have good recall AND precision, or just good recall? 100% recall is trivial to achieve, as is 100% precision. Getting both close to 100% is very difficult.
As a contrary example, here is evidence that deep learning is overfitting. An overfit model is easy to fool since it has incorporated deterministic/stochastic noise. See the following image for an example of overfitting.

Also, see lower ranked answers to this question to understand the problems with an overfit model despite good accuracy on test data.
Some have responded that regularization solves the problem of a large VC dimension.  See this question for further discussion.

"If the map and the terrain disagree, trust the terrain."
It's not really understood why deep learning works as well as it does, but certainly old concepts from learning theory such as VC dimensions appear not to be very helpful.
The matter is hotly debated, see e.g.:

H. W. Lin, M. Tegmark, D. Rolnick, Why does deep and cheap learning work so well?
C. Zhang, S. Bengio, M. Hardt, B. Recht, O. Vinyals, Understanding Deep Learning Requires Rethinking Generalization.
D. Krueger, B. Ballas, S. Jastrzebski, D. Arpit, M. S. Kanwal, T. Maharaj, E. Bengio, A. Fischer, A. Courville, Deep Nets Dont Learn Via Memorization.

Regarding the issue of adversarial examples, the problem was discovered in:

C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, Going deeper with convolutions.

It is further developed in:

I. Goodfellow, J. Shlens, C. Szegedy, Explaining And Harnessing Adversarial Examples.

There is a lot of follow-on work.
Update March 2020. A new hypothesis that appears to explain some of the mismatch between clear over-parameterisation of modern (feed-forward) NNs and good recognition performance is Frankle and Carbin's Lottery Ticket Hypothesis from 2018:

J. Frankle, M. Carbin, The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks.

The claim is that a "randomly-initialised, dense [feed-forward] neural
network contains a subnetwork that is initialised such that when
trained in isolation it can match the test accuracy of the original
network after training for at most the same number of iterations." Regarding the original question, the Lottery Ticket Hypothesis might be understood as saying that:

Training by stochastic gradient descent searches for small subnetworks that work well and deemphasises the rest of the overparameterised network's learning capacity.

The bigger the original network, the more likely it is to contain a small subnetwork with good performance on the task at hand.

This has found empirical support, e.g. in

H. Zhou, J. Lan, R. Liu, J. Yosinski, Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask.

and theoretical support in:

E. Malach, G. Yehudai, S. Shalev-Shwartz, O. Shamir, Proving the Lottery Ticket Hypothesis: Pruning is All You Need.

As far as I'm aware, it has not yet been possible to generalise the Lottery Ticket Hypothesis to recurrent NNs.
Update March 2021. The original 2016 paper Understanding Deep Learning Requires Rethinking Generalization has been updated. Here is the new version:
C. Zhang, S. Bengio, M. Hardt, B. Recht, O. Vinyals, Understanding Deep Learning (Still) Requires Rethinking Generalization.


Question:
Some people say that it's awful that humans eat animals.  They feel that it's barbaric, because you're killing life and then on top of that, you're eating it, and that you should eat vegetation instead.
But isn't vegetation life too?  Personally, I see no difference between animals and veg as all life has cells, dna etc
So my question is, is it possible for humans to live healthy long lives without eating any type of life, i.e no animals, no plants, no cells (dead or alive) etc? If it is possible, how would it be done?

The answer to your question is yes it is certainly possible.
At one time it was thought that there was something special about "organic" chemicals which meant that they could not be artificially synthesised out of fundamental elements. In 1828 Frederick Wöhler synthesised urea (CO(NH2)2) which is often taken as the first demonstration that the organic v inorganic distinction was not a sound one (for more on this see the Wikipedia article on Wöhler synthesis.
As far as we know all essential human nutrients can be synthesised from inorganic ingredients, even complex molecules such as Vitamin B12.
Other contributors have pointed out that organic pathways for synthesising our food have evolved over long periods to be very efficient - at least in the conditions prevailing on Earth. You haven't ruled out copying biochemical pathways using chemicals that are entirely of inorganic origin. Anyone trying to do this seriously could create glucose (for example) by artificially creating enzymes (perhaps via artificial DNA) to do the job. The thing is that we already have self-replicating and repairing machines to do that already (plants).
There might be circumstances when we needed to use artificial synthesis. I can think of two science-fiction stories that deal with this question, the first of which goes into some detail:

The Moon is Hell by John W. Campbell, in which astronauts are stranded on the moon and forced to make food from what they find there.
Technical Error by Arthur C. Clarke, in which a man is accidentally rotated through the fourth dimension. His employers contemplate the difficulty caused by the "handedness" of many biological molecules meaning they would have to artificially synthesise many of his foods.

It may be that a future expedition to Mars (say) might have to think about these things.
A little searching fails to come up with standard inorganic syntheses of glucose and similar substances. The reason for this is almost certainly because it is so easy to use organic inputs. Glucose is easily made by the hydrolysis of starch. Starch is very common and cheap. Even l-glucose is usually made out of organically derived precursors (or sometimes even using d-glucose).
UPDATE: sources etc
One problematic question is: where do you get your input for making nutrients? As others have pointed out, exactly where to draw the line is difficult.
This problem starts in defining what is alive in the first place. Do you count viruses (which can go down to a few thousand base pairs of RNA) or satellite viruses (STobRV has only 359 base pairs) or prions? In a sense these are "just" very large molecules. But then really simple bacteria are not many orders of magnitude more complex. As an aside most systems of ethics that do not permit eating meat do not make an alive/non-alive distinction, choosing some other aspect such as sentience, though Jainism comes close to doing so.
The second problem is, if we reject living things as sources of food, how far removed from those living things are we allowed to get? You say no cells in any state including "dead". That would exclude (say) fruit even though most fruits are expressly created by plants in order to be eaten (and in some cases must be eaten) - something that vegans, jains, fruitarians and others would be happy with eating. If we could use dead material things would be much easier.
But would you also include hydrocarbons (coal, oil, gas) which were once living organisms? If you do, then you are in difficulty because terrestrial carbon is recycled through the biosphere. All CO2 was (to a close approximation) once a part of a living thing. If you take that position then of course you are going to have to go off-planet to find your source chemicals and your problem becomes very much harder.
I was assuming that you were restricting yourself to consuming cells that retain some of their cell structure but had not completely degraded. If that is where you draw the line then there are ample sources of raw materials on earth.
Genetic modification is much more science fiction though not entirely impossible. Some nutrients could be made by humans without much difficulty. Our inability to manufacture vitamin C is down to one missing enzyme (L-gulono-gamma-lactone oxidase) which is present in most vertebrates (I think of mammals only guinea pigs, humans and some bats are unable to synthesise it). You could certainly imagine some very careful genetic modification changing humans so they no longer need to consume vitamin C.
But photosynthesis would be much harder. Chloroplasts (which do the job in most plants) are really a very primitive form of life living in plant cells which may independently reproduce  (and for that reason might be excluded by you - they aren't "cells" but they have membranes). They could easily end up in conflict with our mitochondria (since intracellular conflict between organelles is possible) and you would need to do enormous amounts of work to make human cells co-operate with them properly.
More in keeping with your theme would be adding photosynthetic systems directly to human cells along with a suite of enzymes to manufacture all the things we cannot. That is of course in principle scientifically possible (since plants do it) but much harder than it looks. Living systems are very complicated and small changes can have unexpected consequences. Even very minor genetic modifications are problematic. The human autotroph is likely to be some way off.


Question:
I believe I have a reasonable grasp of complexities like $\mathcal{O}(1)$, $\Theta(n)$ and $\Theta(n^2)$.
In terms of a list, $\mathcal{O}(1)$ is a constant lookup, so it's just getting the head of the list.
$\Theta(n)$ is where I'd walk the entire list once, and $\Theta(n^2)$ is walking the list once for each element in the list.
Is there a similarly intuitive way to grasp $\Theta(\log n)$ other than just knowing it lies somewhere between $\mathcal{O}(1)$ and $\Theta(n)$?

The $\Theta(\log n)$ complexity is usually connected with subdivision. When using lists as an example, imagine a list whose elements are sorted. You can search in this list in $\mathcal{O}(\log n)$ time - you do not actually need to look at each element because of the sorted nature of the list.
If you look at the element in the middle of the list and compare it to the element you search for, you can immediately say whether it lies in the left or right half of the array. Then you can just take this one half and repeat the procedure until you find it or reach a list with 1 item which you trivially compare.
You can see that the list effectively halves each step. That means if you get a list of length $32$, the maximum steps you need to reach one-item list is $5$. If you have a list of $128 = 2^7$ items, you need only $7$ steps, for a list of $1024 = 2^{10}$ you need only $10$ steps etc.
As you can see, the exponent $n$ in $2^n$ always shows the number of steps necessary. Logarithm is used to "extract" exactly this exponent number, for example $\log_2 2^{10} = 10$. It also generalizes to list lengths that are not powers of two long.


Question: Many plants (e.g. roses, palms) can be protected from frost during the winter if shielded with an appropriate coat that can be bought in garden shops. Do plants produce any heat that can be kept inside with these "clothes"?

Cellular respiration in plants is slightly different than in other eukaryotes because the electron transport chain contains an additional enzyme called Alternative Oxidase (AOX). AOX takes some electrons out of the pathway prematurely - basically the energy is used to generate heat instead of ATP.
The exact purpose of AOX in plants is still unclear. Plants will make more AOX in response to cold, wounding, and oxidative stress. We know of at least one plant (skunk cabbage) that exploits this pathway to generate enough heat to melt snow. This link gives a pretty good overview.

(AOX is dear to my heart, since my first 3 years working in a laboratory were spent studying this gene <3)


Question:
Is it possible to kill yourself by holding your breath?

This question is obviously copied from Quora, but I had heard it as a fact that we cannot kill ourselves by holding our breath and I'm looking for a referenced answer.

Short answer
Healthy people cannot hold their breaths until unconsciousness sets in, let alone commit suicide.
Background
According to Parkes (2005), a normal person cannot even hold their breath to unconsciousness, let alone death. Parkes says:

Breath‐holding is a voluntary act, but normal subjects appear unable
  to breath‐hold to unconsciousness. A powerful involuntary mechanism
  normally overrides voluntary breath‐holding and causes the breath that
  defines the breakpoint.

Parkes explains that voluntary breath‐holding does not stop the central respiratory rhythm. Instead, breath holding merely suppresses its expression by voluntarily holding the chest at a certain volume. At the time of writing, no simple explanation for the break point existed. It is known to be caused by partial pressures of blood gases activating the carotid arterial chemoreceptors. They are peripheral sensory neurons that detect changes in chemical concentrations, including low oxygen (hypoxia) and high carbon dioxide (hypercapnia). Both hypoxia and hypercapnia are signs of breath holding and both are detected by the chemoreceptors. These receptors send nerve signals to the vasomotor center of the medulla which eventually overrides the conscious breath holding.
The breaking point can be postponed by large lung inflations, hyperoxia and hypocapnia, and it is shortened by increased metabolic rates.
Reference
- Parkes, Exp Physiol (2006); 91(1): 1-15


Question:
I have several challenging non-convex global optimization problems to solve.  Currently I use MATLAB's Optimization Toolbox (specifically, fmincon() with algorithm='sqp'), which is quite effective.  However, most of my code is in Python, and I'd love to do the optimization in Python as well.  Is there a NLP solver with Python bindings that can compete with fmincon()?  It must

be able to handle nonlinear equality and inequality constraints
not require the user to provide a Jacobian.

It's okay if it doesn't guarantee a global optimum (fmincon() does not). I'm looking for something that robustly converges to a local optimum even for challenging problems, and even if it's slightly slower than fmincon().
I have tried several of the solvers available through OpenOpt and found them to be inferior to MATLAB's fmincon/sqp.
Just for emphasis I already have a tractable formulation and a good solver.  My goal is merely to change languages in order to have a more streamlined workflow.
Geoff points out that some characteristics of the problem may be relevant.  They are:

10-400 decision variables
4-100 polynomial equality constraints (polynomial degree ranges from 1 to about 8)
A number of rational inequality constraints equal to about twice the number of decision variables
The objective function is one of the decision variables

The Jacobian of the equality constraints is dense, as is the Jacobian of the inequality constraints.

I work in a lab that does global optimization of mixed-integer and non-convex problems. My experience with open source optimization solvers has been that the better ones are typically written in a compiled language, and they fare poorly compared to commercial optimization packages.
If you can formulate your problem as an explicit system of equations and need a free solver, your best bet is probably IPOPT, as Aron said. Other free solvers can be found on the COIN-OR web site. To my knowledge, the nonlinear solvers do not have Python bindings provided by the developers; any bindings you find would be third-party. In order to obtain good solutions, you would also have to wrap any nonlinear, convex solver you found in appropriate stochastic global optimization heuristics, or in a deterministic global optimization algorithm such as branch-and-bound. Alternatively, you could use Bonmin or Couenne, both of which are deterministic non-convex optimization solvers that perform serviceably well compared to the state-of-the-art solver, BARON.
If you can purchase a commercial optimization solver, you might consider looking at the GAMS modeling language, which includes several nonlinear optimization solvers. Of particular mention are the interfaces to the solvers CONOPT, SNOPT, and BARON. (CONOPT and SNOPT are convex solvers.) A kludgey solution that I've used in the past is to use the Fortran (or Matlab) language bindings to GAMS to write a GAMS file and call GAMS from Fortran (or Matlab) to calculate the solution of an optimization problem. GAMS has Python language bindings, and a very responsive support staff willing to help out if there's any trouble. (Disclaimer: I have no affiliation with GAMS, but my lab does own a GAMS license.) The commercial solvers should be no worse than fmincon; in fact, I'd be surprised if they weren't a lot better. If your problems are sufficiently small in size, then you may not even need to purchase a GAMS license and licenses to solvers, because an evaluation copy of GAMS may be downloaded from their web site. Otherwise, you would probably want to decide which solvers to purchase in conjunction with a GAMS license. It's worth noting that BARON requires a mixed-integer linear programming solver, and that licenses for the two best mixed-integer linear programming solvers CPLEX and GUROBI are free for academics, so you might be able to get away with just purchasing the GAMS interfaces rather than the interfaces and the solver licenses, which can save you quite a bit of money.
This point bears repeating: for any of the deterministic non-convex optimization solvers I've mentioned above, you need to be able to formulate the model as an explicit set of equations. Otherwise, the non-convex optimization algorithms won't work, because all of them rely on symbolic analysis to construct convex relaxations for branch-and-bound-like algorithms.
UPDATE: One thought that hadn't occurred to me at first was that you could also call the Toolkit for Advanced Optimization (TAO) and PETSc using tao4py and petsc4py, which would have the potential added benefit of easier parallelization, and leveraging familiarity with PETSc and the ACTS tools.
UPDATE #2: Based on the additional information you mentioned, sequential quadratic programming (SQP) methods are going to be your best bet. SQP methods are generally considered more robust than interior point methods, but have the drawback of requiring dense linear solves. Since you care more about robustness than speed, SQP is going to be your best bet. I can't find a good SQP solver out there written in Python (and apparently, neither could Sven Leyffer at Argonne in this technical report). I'm guessing that the algorithms implemented in packages like SciPy and OpenOpt have the basic skeleton of some SQP algorithms implemented, but without the specialized heuristics that more advanced codes use to overcome convergence issues. You could try NLopt, written by Steven Johnson at MIT. I don't have high hopes for it because it doesn't have any reputation that I know of, but Steven Johnson is a brilliant guy who writes good software (after all, he did co-write FFTW). It does implement a version of SQP; if it's good software, let me know.
I was hoping that TAO would have something in the way of a constrained optimization solver, but it doesn't. You could certainly use what they have to build one up; they have a lot of the components there. As you pointed out, though, it'd be much more work for you to do that, and if you're going to that sort of trouble, you might as well be a TAO developer.
With that additional information, you are more likely to get better results calling GAMS from Python (if that's an option at all), or trying to patch up the IPOPT Python interface. Since IPOPT uses an interior point method, it won't be as robust, but maybe Andreas' implementation of an interior point method is considerably better than Matlab's implementation of SQP, in which case, you may not be sacrificing robustness at all. You'd have to run some case studies to know for sure.
You're already aware of the trick to reformulate the rational inequality constraints as polynomial inequality constraints (it's in your book); the reason this would help BARON and some other nonconvex solvers is that it can use term analysis to generate additional valid inequalities that it can use as cuts to improve and speed up solver convergence.
Excluding the GAMS Python bindings and the Python interface to IPOPT, the answer is no, there aren't any high quality nonlinear programming solvers for Python yet. Maybe @Dominique will change that with NLPy.
UPDATE #3: More wild stabs at finding a Python-based solver yielded PyGMO, which is a set of Python bindings to PaGMO, a C++ based global multiobjective optimization solver. Although it was created for multiobjective optimization, it can also be used to single objective nonlinear programming, and has Python interfaces to IPOPT and SNOPT, among other solvers. It was developed within the European Space Agency, so hopefully there's a community behind it. It was also released relatively recently (November 24, 2011).


Question:
I was learning about voltaic cells and came across salt bridges. If the purpose of the salt bridge is only to move electrons from an electrolyte solution to the other, then why can I not use a wire?
Also, will using $\ce{NaCl}$ instead of $\ce{KNO3}$ in making the salt bridge have any effects on voltage/current output of the cell? why?
Plus if it matters, I'm using a Zinc-Copper voltaic cell with a tissue paper soaked in $\ce{KNO3}$ as salt bridge

There's another question related to salt bridges on this site.
The purpose of a salt bridge is not to move electrons from the electrolyte, rather it's to maintain charge balance because the electrons are moving from one-half cell to the other.

The electrons flow from the anode to the cathode.  The oxidation reaction that occurs at the anode generates electrons and positively charged ions.  The electrons move through the wire (and your device, which I haven't included in the diagram), leaving the unbalanced positive charge in this vessel.  In order to maintain neutrality, the negatively charged ions in the salt bridge will migrate into the anodic half cell.  A similar (but reversed) situation is found in the cathodic cell, where $\ce{Cu^{2+}}$ ions are being consumed, and therefore electroneutrality is maintained by the migration of $\ce{K+}$ ions from the salt bridge into this half cell.
Regarding the second part of your question, it is important to use a salt with inert ions in your salt bridge. In your case, you probably won't notice a difference between $\ce{NaCl}$ and $\ce{KNO3}$ since the $\ce{Cu^{2+}}$ and $\ce{Zn^{2+}}$ salts of $\ce{Cl-}$ and $\ce{NO3-}$ are soluble. There will be a difference in the liquid junction potential, but that topic is a bit advanced for someone just starting out with voltaic/galvanic cells.