Question:
Today a friend's six year old sister asked me the question "why don't people on the other side of the earth fall off?". I tried to explain that the Earth is a huge sphere and there's a special force called "gravity" that tries to attract everything to the center of the Earth, but she doesn't seem to understand it. I also made some attempts using a globe, saying that "Up" and "Down" are all local perspective and people on the other side of the Earth feel they're on the top, but she still doesn't get it.
How can I explain the concept of gravity to a six year old in a simple and meaningful way?

Having my own 6-year-old and having successfully explained this, here's my advice from experience:

Don't try to explain gravity as a mysterious force. It doesn't make sense to most adults (sad, but true! talk to non-physicists about it and you'll see), it won't make sense to a 6yo.
The reason this won't work is that it requires inference from general principles to specific applications, plus it requires advanced abstract thinking to even grasp the concept of invisible forces. Those are not skills a 6-year-old has at their fingertips. Most things they're figuring out right now is piecemeal and they won't start fitting their experiences to best-fit conscious models of reality for a few years yet.
Do exploit 6-year-old's tendency to take descriptions of actions-that-happen at face value as simple piecemeal facts.

Stuff pulls other stuff to itself. When you have a lot of stuff, it pulls other things a lot. The bigger things pull the smaller things to them.

Them having previously understood the shape of the solar system and a loose grasp of the fact of orbits (not how they work—that's a different piece—just that planets and moons move in "circular" tracks around heavier things like the Sun and Earth) may be useful before embarking on these parts of the conversation. I'm not sure, but that was a thing my 6yo already had started to grasp at this point.
These conversations were also mixed in with our conversations about how Earth formed from debris, and how the pull was involved in making that happen, and how it made the pull more and more. So, I can't really separate out that background; it may also help/be necessary.
Don't try to correct a 6-year-old's confusion about up and down being relative, but use it instead.

There's a lot of Earth under us, and it pulls us down when we jump. If we jumped off the side, it would pull us back sideways. If we fell off the bottom, it would pull us back up.

You can follow this up later with a Socratic dialogue about the relative nature of up and down, but don't muddy the waters with that immediately. That won't have any purchase until they accept the fact that Earth will pull you "back up" if you fall off.
Build it up over a series of conversations. They won't get it the first time, or the tenth, but pieces of it will stick.
Don't try to instill a grasp of the overall working model. If you can successfully give them some single, disconnected facts that they actually believe, putting them together will happen as they age and mature and get more exposure to this stuff.

All this is assuming a decently smart but not prodigious child, of course. (A 6-year-old prodigy can probably grasp a lay adult's model of gravity, but if that's who you're dealing with then you don't need to adjust your teaching.)
For some more context, this was also after my child's class started experimenting with magnets at school. I was inspired to attempt to explain gravity when my kid told me that trees didn't float off into space because the Earth was a giant magnet. (True! But not why trees don't float away.) Comparing gravity and magnetism might help, to give them an example of invisible pull that they can feel, but it might just confuse the subject a lot too since I had a lot of work (over multiple conversations) to convince my own that trees aren't sticking to the ground because of magnetism, even if the Earth is a giant magnet.
And, a final piece of advice that's incidental, but can help:

Once you've had a few of these conversations, play Kerbal Space Program while they watch. (Again, this comes from experience. My kid loves to watch KSP.) Seeing a practical example of gravity at work in it natural environment will go a long way to cementing the previous conversations. It may sound like a sign-off joke, but seeing a system moving and being manipulated makes a huge difference to a young child's comprehension, because it is no longer abstract or requires building mental abstractions to grasp, like showing them a globe does.


Question:
I currently find Harvard's RESTful API for ExAC extremely useful and I was hoping that a similar resource is available for Gnomad?
Does anyone know of a public access API for Gnomad or possibly any plans to integrate Gnomad into the Harvard API?

As far as I know, no but the vcf.gz files are behind a http server that supports Byte-Range, so you can use tabix or any related API:
$ tabix " "22:17265182-17265182"
22  17265182    .   A   T   762.04  PASS    AC=1;AF=4.78057e-06;AN=209180;BaseQRankSum=-4.59400e+00;ClippingRankSum=2.18000e+00;DP=4906893;FS=1.00270e+01;InbreedingCoeff=4.40000e-03;MQ=3.15200e+01;MQRankSum=1.40000e+00;QD=1.31400e+01;ReadPosRankSum=2.23000e-01;SOR=9.90000e-02;VQSLOD=-5.12800e+00;VQSR_culprit=MQ;GQ_HIST_ALT=0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|1;DP_HIST_ALT=0|0|0|0|0|0|0|0|0|0|0|1|0|0|0|0|0|0|0|0;AB_HIST_ALT=0|0|0|0|0|0|0|0|0|0|0|0|1|0|0|0|0|0|0|0;GQ_HIST_ALL=1591|589|120|301|650|589|1854|2745|1815|4297|5061|2921|10164|1008|6489|1560|7017|457|6143|52950;DP_HIST_ALL=2249|1418|6081|11707|16538|9514|28624|23829|7391|853|95|19|1|0|0|1|0|1|0|0;AB_HIST_ALL=0|0|0|0|0|0|0|0|0|0|0|0|1|0|0|0|0|0|0|0;AC_AFR=0;AC_AMR=0;AC_ASJ=0;AC_EAS=0;AC_FIN=1;AC_NFE=0;AC_OTH=0;AC_SAS=0;AC_Male=1;AC_Female=0;AN_AFR=11994;AN_AMR=31324;AN_ASJ=7806;AN_EAS=13112;AN_FIN=20076;AN_NFE=94516;AN_OTH=4656;AN_SAS=25696;AN_Male=114366;AN_Female=94814;AF_AFR=0.00000e+00;AF_AMR=0.00000e+00;AF_ASJ=0.00000e+00;AF_EAS=0.00000e+00;AF_FIN=4.98107e-05;AF_NFE=0.00000e+00;AF_OTH=0.00000e+00;AF_SAS=0.00000e+00;AF_Male=8.74386e-06;AF_Female=0.00000e+00;GC_AFR=5997,0,0;GC_AMR=15662,0,0;GC_ASJ=3903,0,0;GC_EAS=6556,0,0;GC_FIN=10037,1,0;GC_NFE=47258,0,0;GC_OTH=2328,0,0;GC_SAS=12848,0,0;GC_Male=57182,1,0;GC_Female=47407,0,0;AC_raw=1;AN_raw=216642;AF_raw=4.61591e-06;GC_raw=108320,1,0;GC=104589,1,0;Hom_AFR=0;Hom_AMR=0;Hom_ASJ=0;Hom_EAS=0;Hom_FIN=0;Hom_NFE=0;Hom_OTH=0;Hom_SAS=0;Hom_Male=0;Hom_Female=0;Hom_raw=0;Hom=0;POPMAX=FIN;AC_POPMAX=1;AN_POPMAX=20076;AF_POPMAX=4.98107e-05;DP_MEDIAN=58;DREF_MEDIAN=5.01187e-84;GQ_MEDIAN=99;AB_MEDIAN=6.03448e-01;AS_RF=9.18451e-01;AS_FilterStatus=PASS;CSQ=T|missense_variant|MODERATE|XKR3|ENSG00000172967|Transcript|ENST00000331428|protein_coding|4/4||ENST00000331428.5:c.707T>A|ENSP00000331704.5:p.Phe236Tyr|810|707|236|F/Y|tTc/tAc||1||-1||SNV|1|HGNC|28778|YES|||CCDS42975.1|ENSP00000331704|Q5GH77||UPI000013EFAE||deleterious(0)|benign(0.055)|hmmpanther:PTHR14297&hmmpanther:PTHR14297:SF7&Pfam_domain:PF09815||||||||||||||||||||||||||||||,T|regulatory_region_variant|MODIFIER|||RegulatoryFeature|ENSR00000672806|TF_binding_site|||||||||||1||||SNV|1||||||||||||||||||||||||||||||||||||||||||||,T|regulatory_region_variant|MODIFIER|||RegulatoryFeature|ENSR00001729562|CTCF_binding_site|||||||||||1||||SNV|1||||||||||||||||||||||||||||||||||||||||||||

UPDATE: 2019: the current server for gnomad doesn't support Byte-Range requests.


Question:
I am trying to understand the benefits of joint genotyping and would be grateful if someone could provide an argument (ideally mathematically) that would clearly demonstrate the benefit of joint vs. single-sample genotyping.
This is what I've gathered from other resources (Biostars, GATK forums, etc.)

Joint-genotyping helps control FDR because errors from individually genotyped samples are added up, and amplified when merging call-sets (by Heng Li on

If someone understands this, can you please clarify what is the difference on the overall FDR rate between the two scenarios (again, with an example ideally)

Greater sensitivity for low-frequency variants - By sharing information across all samples, joint calling makes it possible to “rescue” genotype calls at sites where a carrier has low coverage but other samples within the call set have a confident variant at that location. (from

I don't understand how the presence of a confidently called variant at the same locus in another individual can affect the genotyping of an individual with low coverage. Is there some valid argument that allows one to consider reads from another person as evidence of a particular variant in a third person? What are the assumptions for such an argument? What if that person is from a different population with entirely different allele frequencies for that variant?
Having read several of the papers (or method descriptions) that describe the latest haplotype-aware SNP calling methods (HaplotypeCaller, freebayes, Platypus) the overall framework seems to be:

Establish a prior on the allele frequency distribution at a site of interest using one (or combination) of: non-informative prior, population genetics model-based prior like Wright Fisher, prior based on established variation patterns like dbSNP, ExAC, or gnomAD.

Build a list of plausible haplotypes in a region around the locus of interest using local assembly.

Select haplotype with highest likelihood based on prior and reads data and infer the locus genotype accordingly.

At which point(s) in the above procedure can information between samples be shared or pooled? Should one not trust the AFS from a large-scale resource like gnomAD much more than the distribution obtained from other samples that are nominally party of the same "cohort" but may have little to do with each other because of different ancestry, for example?
I really want to understand the justifications and benefits offered by multi-sample genotyping and would appreciate your insights.

Say you are sequencing to 2X coverage. Suppose at a site, sample S has one reference base and one alternate base. It is hard to tell if this is a sequencing error or a heterozygote. Now suppose you have 1000 other samples, all at 2X read depth. One of them has two ALT bases; 10 of them have one REF and one ALT. It is usually improbable that all these samples have the same sequencing error. Then you can assert sample S has a het. Multi-sample calling helps to increase the sensitivity of not so rare SNPs. Note that what matters here is the assumption of error independency. Ancestry only has a tiny indirect effect.
Multi-sample calling penalizes very rare SNPs, in particular singletons. When you care about variants only, this is for good. Naively combining single-sample calls yields a higher error rate. Multi-sample calling also helps variant filtering at a later stage. For example, for a sample sequenced to 30X coverage, you would not know if a site at 45X depth is caused by a potential CNV/mismapping or by statistical fluctuation. When you see 1000 30X samples at 45X depth, you can easily know you are looking at a CNV/systematic mismapping. Multiple samples enhance most statistical signals.
Older methods pool all BAMs when calling variants. This is necessary because a single low-coverage sample does not have enough data to recover hidden INDELs. However, this strategy is not that easy to massively parallelized; adding a new sample triggers re-calling, which is very expensive as well. As we are mostly doing high-coverage sequencing these days, the old problem with INDEL calling does not matter now. GATK has this new single-sample calling pipeline where you combine per-sample gVCFs at a later stage. Such sample combining strategy is perhaps the only sensible solution when you are dealing with 100k samples.
The so-called haplotype based variant calling is a separate question. This type of approach helps to call INDELs, but is not of much relevance to multi-sample calling. Also, of the three variant callers in your question, only GATK (and Scalpel which you have not mentioned) use assembly at large. Freebayes does not. Platypus does but only to a limited extent and does not work well in practice.
I guess what you really want to talk about is imputation based calling. This approach further improves sensitivity with LD. With enough samples, you can measure the LD between two positions. Suppose at position 1000, you see one REF read and no ALT reads; at position 1500, you see one REF read and two ALT reads. You would not call any SNPs at position 1000 even given multiple samples. However, when you know the two positions are strongly linked and the dominant haplotypes are REF-REF and ALT-ALT, you know the sample under investigation is likely to have a missing ALT allele. LD transfers signals across sites and enhances the power to make correct genotyping calls. Nonetheless, as we are mostly doing high-coverage sequencing nowadays, imputation based methods only have a minor effect and are rarely applied.


Question: I'd be tempted to call nipples in men vestigial, but that suggests they have no modern function. They do have a function, of course, but only in women. So why do men (and all male mammals) have them?

I believe it is for this reason: the female body plan is the default one. Males are a variation upon that, in humans at least. Nipples are part of the basic body plan. For a man to not have them, he would need to actively evolve something that would prevent nipples from developing. There is no selective pressure for the development of such a thing, so it hasn't happened. Keep in mind that the code for the general body plan is shared between males and females. The Y chromosome modifies the development of that body plan so the person becomes male.


Question:
A lot of the organometallics are rather... interesting compounds to work with. The most famous (among those who care, anyway) is tert-butyllithium or t-BuLi. It is the textbook example of a pyrophoric substance, demonstrated to pretty much every chemistry major as an air-sensitive chemical requiring special handling (Syringe and cannula transfers, gas-tight septa, argon/nitrogen blankets, that kind of thing). Despite the inherent hazards of the stuff, it's widely used in the industry (hence widespread notoriety) to add butyl groups to an organic molecule (most organometallics are useful for this kind of carbon-carbon bond formation), and you can buy it by the - airtight - gallon canister.
There are, surprisingly (except to most organic chemists), even more dangerous compounds in the average organic chemistry lab. Along the pyrophoric line, many of the multi-methyl-metallics are violently pyrophoric (even compared to BuLi), including trimethylaluminum, dimethylzinc, and dimethylmagnesium. All of these are also extremely poisonous (anything these compounds can do to your wonder-drug-in-progress, they can also do to various key structures in your own body), with dubious honors reserved for dimethylmercury. Because many of these, especially the organoalkaline compounds, react violently with water, they also automatically rate at least a 3 on the "reactivity" scale.
When leads me to wonder, because I wonder about these things from the safety of my office chair in a completely unrelated field; just how bad can it get? Specifically, is there any compound with an accepted use in laboratory or industrial application that is nasty enough to max out the entire NFPA-704 diamond? The closest I can find is trimethylaluminum, at a 3-4-3 (heath-fire-reactivity). T-BuLi is a 3-3-4. I can't find NFPA data on straight diazomethane (which may be because nobody in their right mind ever works with the stuff in its pure gaseous form; it's always used in a dilute diethyl ether solution, and even then is never sold or shipped that way) but it would probably be a finalist, as the gas is acutely toxic, autoignites at room temperature, and detonates on standing (something for everyone!). I'm thinking that these two inorganic families - light alkyl-metallics and organo-polyazides - would be the most likely candidates to produce a compound so toxic, so flammable, and so readily reactive, yet so interesting to chemistry, that the NFPA would see fit to rate it, and would give it highest honors.

Answering my own question based on the comments, tert-butyl-hydroperoxide is at least one such chemical. As stated on this MSDS from a government website, it's a 4-4-4, with additional special warning of being a strong oxidizer. The only thing that it does not do that could make the 704 diamond any worse is react strongly with water. It is in fact water soluble, though marginally, preferring to float on top (and therefore traditional water-based fire suppression is ineffective, but foam/CO2 will work).
If anyone else can find a chemical that, in a form that is used in the lab or industrially, is a 4-4-4 that is a strong oxidizer and reacts strongly with water, that's pretty much "as bad as it gets" and they'll get the check.


Question:
We have a random experiment with different outcomes forming the sample space $\Omega,$ on which we look with interest at certain patterns, called events $\mathscr{F}.$ Sigma-algebras (or sigma-fields) are made up of events to which a probability measure $\mathbb{P}$ can be assigned. Certain properties are fulfilled, including the inclusion of the null set $\varnothing$ and the entire sample space, and an algebra that describes unions and intersections with Venn diagrams.
Probability is defined as a function between the $\sigma$-algebra and the interval $[0,1]$.
 Altogether, the triple $(\Omega, \mathscr{F}, \mathbb{P})$ forms a probability space.
Could someone explain in plain English why the probability edifice would collapse if we didn't have a $\sigma$-algebra? They are just wedged in the middle with that impossibly calligraphic "F". I trust they are necessary; I see that an event is different from an outcome, but what would go awry without a $\sigma$-algebras?

The question is: In what type of probability problems the definition of a probability space including a $\sigma$-algebra becomes a necessity?

This online document on the Dartmouth University website provides a plain English accessible explanation. The idea is a spinning pointer rotating counterclockwise on a circle of unit perimeter:

We begin by constructing a spinner, which consists of a circle of unit
  circumference and a pointer as shown in [the] Figure. We pick a point
  on the circle and label it $0$, and then label every other point on the
  circle with the distance, say $x$, from $0$ to that point, measured
  counterclockwise. The  experiment  consists  of spinning the pointer and
  recording the label of the point at the tip of the pointer. We let the random
  variable $X$ denote the value of this outcome.  The sample space is
  clearly the interval $[0,1)$.  We would like to construct a
  probability model in which each outcome is equally likely to occur. If
  we proceed as we did [...] for experiments with a finite number of
  possible outcomes,  then we must assign the probability $0$ to each
  outcome,  since otherwise,  the  sum  of  the  probabilities,  over
  all  of  the  possible  outcomes,  would not equal 1.  (In fact,
  summing an uncountable number of real numbers is a tricky business;
  in  particular,  in  order  for  such  a  sum  to  have  any  meaning,
  at  most countably many of the summands can be different than $0$.)
  However, if all of the assigned probabilities are $0$, then the sum is
  $0$, not $1$, as it should be.

So if we assigned to each point any probability, and given that there is an (uncountably) infinity number of points, their sum would add up to $> 1$.

To Xi'an's first point: When you're talking about $\sigma$-algebras, you're asking about measurable sets, so unfortunately any answer must focus on measure theory. I'll try to build up to that gently, though.
A theory of probability admitting all subsets of uncountable sets will break mathematics
Consider this example. Suppose you have a unit square in $\mathbb{R}^2$, and you're interested in the probability of randomly selecting a point that is a member of a specific set in the unit square. In lots of circumstances, this can be readily answered based on a comparison of areas of the different sets. For example, we can draw some circles, measure their areas, and then take the probability as the fraction of the square falling in the circle. Very simple.
But what if the area of the set of interest is not well-defined?
If the area is not well-defined, then we can reason to two different but completely valid (in some sense) conclusions about what the area is. So we could have $P(A)=1$ on the one hand and $P(A)=0$ on the other hand, which implies $0=1$. This breaks all of math beyond repair. You can now prove $5<0$ and a number of other preposterous things. Clearly this isn't too useful.
$\boldsymbol{\sigma}$-algebras are the patch that fixes math
What is a $\sigma$-algebra, precisely? It's actually not that frightening. It's just a definition of which sets may be considered as events. Elements not in $\mathscr{F}$ simply have no defined probability measure. Basically, $\sigma$-algebras are the "patch" that lets us avoid some pathological behaviors of mathematics, namely non-measurable sets.
The three requirements of a $\sigma$-field can be considered as consequences of what we would like to do with probability:
A $\sigma$-field is a set that has three properties:

Closure under countable unions.
Closure under countable intersections.
Closure under complements.

The countable unions and countable intersections components are direct consequences of the non-measurable set issue. Closure under complements is a consequence of the Kolmogorov axioms: if $P(A)=2/3$, $P(A^c)$ ought to be $1/3$. But without (3), it could happen that $P(A^c)$ is undefined. That would be strange. Closure under complements and the Kolmogorov axioms let us to say things like $P(A\cup A^c)=P(A)+1-P(A)=1$.
Finally, We are considering events in relation to $\Omega$, so we further require that $\Omega\in\mathscr{F}$
Good news: $\boldsymbol{\sigma}$-algebras are only strictly necessary for uncountable sets
But! There's good news here, also. Or, at least, a way to skirt the issue. We only need $\sigma$-algebras if we're working in a set with uncountable cardinality. If we restrict ourselves to countable sets, then we can take $\mathscr{F}=2^\Omega$ the power set of $\Omega$ and we won't have any of these problems because for countable $\Omega$, $2^\Omega$ consists only of measurable sets. (This is alluded to in Xi'an's second comment.) You'll notice that some textbooks will actually commit a subtle sleight-of-hand here, and only consider countable sets when discussing probability spaces.
Additionally, in geometric problems in $\mathbb{R}^n$, it's perfectly sufficient to only consider $\sigma$-algebras composed of sets for which the $\mathcal{L}^n$ measure is defined. To ground this somewhat more firmly, $\mathcal{L}^n$ for $n=1,2,3$ corresponds to the usual notions of length, area and volume. So what I'm saying in the previous example is that the set needs to have a well-defined area for it to have a geometric probability assigned to it. And the reason is this: if we admit non-measureable sets, then we can end up in situations where we can assign probability 1 to some event based on some proof, and probability 0 to the same event event based on some other proof.
But don't let the connection to uncountable sets confuse you! A common misconception that $\sigma$-algebras are countable sets. In fact, they may be countable or uncountable. Consider this illustration: as before, we have a unit square. Define $$\mathscr{F}=\text{All subsets of the unit square with defined $\mathcal{L}^2$ measure}.$$ You can draw a square $B$ with side length $s$ for all $s \in (0,1)$, and with one corner at $(0,0)$. It should be clear that this square is a subset of the unit square. Moreover, all of these squares have defined area, so these squares are elements of $\mathscr{F}$. But it should also be clear that there are uncountably many squares $B$: the number of such squares is uncountable, and each square has defined Lebesgue measure.
So as a practical matter, simply making that observation is often enough to make the observation that you only consider Lebesgue-measurable sets to gain headway against the problem of interest.
But wait, what's a non-measurable set?
I'm afraid I can only shed a little bit of light on this myself. But the Banach-Tarski paradox (sometimes the "sun and pea" paradox) can help us some:

Given a solid ball in 3‑dimensional space, there exists a decomposition of the ball into a finite number of disjoint subsets, which can then be put back together in a different way to yield two identical copies of the original ball. Indeed, the reassembly process involves only moving the pieces around and rotating them, without changing their shape. However, the pieces themselves are not "solids" in the usual sense, but infinite scatterings of points. The reconstruction can work with as few as five pieces.
A stronger form of the theorem implies that given any two "reasonable" solid objects (such as a small ball and a huge ball), either one can be reassembled into the other. This is often stated informally as "a pea can be chopped up and reassembled into the Sun" and called the "pea and the Sun paradox".1

So if you're working with probabilities in $\mathbb{R}^3$ and you're using the geometric probability measure (the ratio of volumes), you want to work out the probability of some event. But you'll struggle to define that probability precisely, because you can rearrange the sets of your space to change volumes!
If probability depends on volume, and you can change the volume of the set to be the size of the sun or the size of a pea, then the probability will also change. So no event will have a single probability ascribed to it. Even worse, you can rearrange $S\in\Omega$ such that the volume of $S$ has $V(S)>V(\Omega)$, which implies that the geometric probability measure reports a probability $P(S)>1$, in flagrant violation of the Kolmogorov axioms which require that probability has measure 1.
To resolve this paradox, one could make one of four concessions:

The volume of a set might change when it is rotated.
The volume of the union of two disjoint sets might be different from the sum of their volumes.
The axioms of Zermelo–Fraenkel set theory with the axiom of Choice (ZFC) might have to be altered.
Some sets might be tagged "non-measurable", and one would need to check whether a set is "measurable" before talking about its volume.

Option (1) doesn't help use define probabilities, so it's out. Option (2) violates the second Kolmogorov axiom, so it's out. Option (3) seems like a terrible idea because ZFC fixes so many more problems than it creates. But option (4) seems attractive: if we develop a theory of what is and is not measurable, then we will have well-defined probabilities in this problem! This brings us back to measure theory, and our friend the $\sigma$-algebra.


Question:
I'm trying to download three WGS datasets from the SRA that are each between 60 and 100GB in size.  So far I've tried:

Fetching the .sra files directly from NCBI's ftp site
Fetching the .sra files directly using the aspera command line (ascp)
Using the SRA toolkit's fastqdump and samdump tools

It's excruciatingly slow.  I've had three fastqdump processes running in parallel now for approximately 18 hours.  They're running on a large AWS instance in the US east (Virginia) region, which I figure is about as close to NCBI as I can get.  In 18 hours they've downloaded a total of 33GB of data.  By my calculation that's ~500kb/s.  They do appear to still be running - the fastq files continue to grow and their timestamps continue to update.
At this rate it's going to take me days or weeks just to download the datasets.  Surely the SRA must be capable of moving data at higher rates that this?  I've also looked, and unfortunately the datasets I'm interested have not been mirrored out to ENA or the Japanese archive, so it looks like I'm stuck working with the SRA.
Is there a better way to fetch this data that wouldn't take multiple days?

Proximity to NCBI may not necessarily give you the fastest transfer speed. AWS may be deliberately throttling the Internet connection to limit the likelihood that people will use it for undesirable things. There's a chance that a home network might be faster, but you're likely to get the fastest connection to NCBI by using an academic system that is linked to NCBI via a research network.
Another possibility is using Aspera for downloads. This is unlikely to help if bandwidth is being throttled, but it might help if there's a bit of congestion through the regular methods:

NCBI also has an online book about best practises for downloading data from their servers.
On a related note, just in case someone sees this and EBI/ENA is an option, there's a great guide for how to do file transfer using Aspera on the EBI web site:

Your command should look similar to this on Unix:

ascp -QT -l 300m -i <aspera connect installation directory>/etc/asperaweb_id_dsa.openssh era-fasp@fasp.sra.ebi.ac.uk:<file or files to download> <download location>

In my case, I've just started downloading some files from a MinION sequencing run. The estimated completion time via standard FTP was 12 hours for about 32GB of data; ascp has reduced that estimated download time to about an hour. Here's the command I used for downloading:
ascp -QT -l 300m -i ~/.aspera/connect/etc/asperaweb_id_dsa.openssh era-fasp@fasp.sra.ebi.ac.uk:/vol1/ERA932/ERA932268/oxfordnanopore_native/20160804_Mock.tar.gz .


Question: I'm interested working with the medication information provided by the UK Biobank.  In order to get these into a usable form I would like to map them to ATC codes.  Since many of the drugs listed in the data showcase include dosage information, doing an exact string match between drug names is not very effective.  I've considered using something like fuzzywuzzy to do string matching between the medications in the data showcase and the ATC drug names but validating the matches could still be a laborious process.  Does anyone know of a tool that can match drug names to ATC codes or some other drug ontology?  If not, maybe there's a better way to do it that I haven't thought of.

The CART tool let's you upload a set of names and map them (optionally in a fuzzy way) to STITCH 4 identifiers, and then use those to map to ATC codes (using the chemicals sources download file). It's a bit indirect, and I'm not sure what CART will do with the dosage info you mention.


Question:
A friend of mine was looking over the definition of pH and was wondering if it is possible to have a negative pH. From the equation below, it certainly seems mathematically possible—if you have a $1.1$ (or something $\gt 1$) molar solution of $\ce{H+}$ ions:
$$\text{pH} = -\log([\ce{H+}])$$
(Where $[\ce{X}]$ denotes the concentration of $\ce{X}$ in $\frac{\text{mol}}{\text{L}}$.)
If $[\ce{H+}] = 1.1\ \frac{\text{mol}}{\text{L}}$, then $\mathrm{pH} = -\log(1.1) \approx -0.095 $
So, it is theoretically possible to create a substance with a negative pH. But, is it physically possible (e.g. can we create a 1.1 molar acid in the lab that actually still behaves consistently with that equation)?

One publication for you: “Negative pH Does Exist”, K. F. Lim, J. Chem. Educ. 2006, 83, 1465. Quoting the abstract in full:

The misconception that pH lies between 0 and 14 has been perpetuated in popular-science books, textbooks, revision guides, and reference books.

The article text provides some counterexamples:

For example, commercially available concentrated HCl solution (37% by mass) has $\mathrm{pH} \approx -1.1$, while saturated NaOH solution has $\mathrm{pH} \approx 15.0$.


Question:
Disclaimer: I'm not a statistician but a software engineer. Most of my knowledge in statistics comes from self-education, thus I still have many gaps in understanding concepts that may seem trivial for other people here. So I would be very thankful if answers included less specific terms and more explanation. Imagine that you are talking to your grandma :)
I'm trying to grasp the nature of beta distribution – what it should be used for and how to interpret it in each case. If we were talking about, say, normal distribution, one could describe it as arrival time of a train: most frequently it arrives just in time, a bit less frequently it is 1 minute earlier or 1 minute late and very rarely it arrives with difference of 20 minutes from the mean. Uniform distribution describes, in particular, chance of each ticket in lottery. Binomial distribution may be described with coin flips and so on. But is there such intuitive explanation of beta distribution?
Let's say, $\alpha=.99$ and $\beta=.5$. Beta distribution $B(\alpha, \beta)$ in this case looks like this (generated in R):

But what does it actually mean? Y-axis is obviously a probability density, but what is on the X-axis?
I would highly appreciate any explanation, either with this example or any other.

The short version is that the Beta distribution can be understood as representing a distribution of probabilities, that is, it represents all the possible values of a probability when we don't know what that probability is. Here is my favorite intuitive explanation of this:
Anyone who follows baseball is familiar with batting averages—simply the number of times a player gets a base hit divided by the number of times he goes up at bat (so it's just a percentage between 0 and 1). .266 is in general considered an average batting average, while .300 is considered an excellent one.
Imagine we have a baseball player, and we want to predict what his season-long batting average will be. You might say we can just use his batting average so far- but this will be a very poor measure at the start of a season! If a player goes up to bat once and gets a single, his batting average is briefly 1.000, while if he strikes out, his batting average is 0.000. It doesn't get much better if you go up to bat five or six times- you could get a lucky streak and get an average of 1.000, or an unlucky streak and get an average of 0, neither of which are a remotely good predictor of how you will bat that season.
Why is your batting average in the first few hits not a good predictor of your eventual batting average? When a player's first at-bat is a strikeout, why does no one predict that he'll never get a hit all season? Because we're going in with prior expectations. We know that in history, most batting averages over a season have hovered between something like .215 and .360, with some extremely rare exceptions on either side. We know that if a player gets a few strikeouts in a row at the start, that might indicate he'll end up a bit worse than average, but we know he probably won't deviate from that range.
Given our batting average problem, which can be represented with a binomial distribution (a series of successes and failures), the best way to represent these prior expectations (what we in statistics just call a prior) is with the Beta distribution- it's saying, before we've seen the player take his first swing, what we roughly expect his batting average to be. The domain of the Beta distribution is (0, 1), just like a probability, so we already know we're on the right track, but the appropriateness of the Beta for this task goes far beyond that.
We expect that the player's season-long batting average will be most likely around .27, but that it could reasonably range from .21 to .35. This can be represented with a Beta distribution with parameters $\alpha=81$ and $\beta=219$:
curve(dbeta(x, 81, 219))

I came up with these parameters for two reasons:

The mean is $\frac{\alpha}{\alpha+\beta}=\frac{81}{81+219}=.270$
As you can see in the plot, this distribution lies almost entirely within (.2, .35)- the reasonable range for a batting average.

You asked what the x axis represents in a beta distribution density plot—here it represents his batting average. Thus notice that in this case, not only is the y-axis a probability (or more precisely a probability density), but the x-axis is as well (batting average is just a probability of a hit, after all)! The Beta distribution is representing a probability distribution of probabilities.
But here's why the Beta distribution is so appropriate. Imagine the player gets a single hit. His record for the season is now 1 hit; 1 at bat. We have to then update our probabilities- we want to shift this entire curve over just a bit to reflect our new information. While the math for proving this is a bit involved (it's shown here), the result is very simple. The new Beta distribution will be:
$\mbox{Beta}(\alpha_0+\mbox{hits}, \beta_0+\mbox{misses})$
Where $\alpha_0$ and $\beta_0$ are the parameters we started with- that is, 81 and 219. Thus, in this case, $\alpha$  has increased by 1 (his one hit), while $\beta$ has not increased at all (no misses yet). That means our new distribution is $\mbox{Beta}(81+1, 219)$, or:
curve(dbeta(x, 82, 219))

Notice that it has barely changed at all- the change is indeed invisible to the naked eye! (That's because one hit doesn't really mean anything).
However, the more the player hits over the course of the season, the more the curve will shift to accommodate the new evidence, and furthermore the more it will narrow based on the fact that we have more proof. Let's say halfway through the season he has been up to bat 300 times, hitting 100 out of those times. The new distribution would be $\mbox{Beta}(81+100, 219+200)$, or:
curve(dbeta(x, 81+100, 219+200))

Notice the curve is now both thinner and shifted to the right (higher batting average) than it used to be- we have a better sense of what the player's batting average is.
One of the most interesting outputs of this formula is the expected value of the resulting Beta distribution, which is basically your new estimate. Recall that the expected value of the Beta distribution is $\frac{\alpha}{\alpha+\beta}$. Thus, after 100 hits of 300 real at-bats, the expected value of the new Beta distribution is $\frac{81+100}{81+100+219+200}=.303$- notice that it is lower than the naive estimate of $\frac{100}{100+200}=.333$, but higher than the estimate you started the season with ($\frac{81}{81+219}=.270$). You might notice that this formula is equivalent to adding a "head start" to the number of hits and non-hits of a player- you're saying "start him off in the season with 81 hits and 219 non hits on his record").
Thus, the Beta distribution is best for representing a probabilistic distribution of probabilities: the case where we don't know what a probability is in advance, but we have some reasonable guesses.


Question: In my computational science PhD program, we are working almost exclusively in C++ and Fortran. It seems like some professors prefer one over the other. I am wondering which one is 'better' or if one is better than the other in a certain circumstance.

As so often, the choice depends on (1) the problem you are trying to solve, (2) the skills you have, and (3) the people you work with (unless it's a solo project). I'll leave (3) aside for the moment because it depends on everyone's individual situation.
Problem dependence: Fortran excels at array processing. If your problem can be described in terms of simple data structures and in particular arrays, Fortran is well adapted. Fortran programmers end up using arrays even in non-obvious cases (e.g. for representing graphs). C++ is better suited for complex and highly dynamic data structures.
Skill dependence: it takes a lot more programming experience to write good C++ programs than to write good Fortran programs. If you start out with little programming experience and only have so much time to learn that aspect of your job, you probably get a better return on investment learning Fortran than learning C++. Assuming, of course, that your problem is suited to Fortran.
However, there's more to programming than just Fortran and C++. I'd recommend to anyone going into computational science to start with a dynamic high-level language such as Python. Always remember that your time is more valuable than CPU time!


Question: What makes dimerization possible in $\ce{AlCl3}$? Are there 3c-2e bonds in $\ce{Al2Cl6}$ as there are in $\ce{B2H6}$?

Introduction
The bonding situation in $\ce{(AlCl3)2}$ and $\ce{(BCl3)2}$ is nothing trivial and the reason why aluminium chloride forms dimers, while boron trichloride does not, cannot only be attributed to size.
In order to understand this phenomenon we need to look at both, the monomers and the dimers, and compare them to each other.
Understanding the respective bonding situation of the monomers, is key to understand which deficiencies lead to dimerisations.
Computational details
Since I was unable to find any compelling literature on the subject, I ran some calculations of my own. I used the DF-M06L/def2-TZVPP for geometry optimisations. Each structure has been optimised to a local minimum in their respective symmetry restrictions, i.e. $D_\mathrm{3h}$ for the monomers and $C_\mathrm{2v}$ for the dimers.
Analyses with the Natural Bond Orbital model (NBO6 program) and the Quantum Theory of Atoms in Molecules (QTAIM, MultiWFN) have been run on single point energy calculations at the M06/def2-QZVPP//DF-M06-L/def2-TZVPP level of theory.
A rudimentary energy decomposition analysis has been done on that level, too.
Energy decomposition analysis
The dissociation energy of the dimers $\ce{(XY3)2}$ to the monomers $\ce{XY3}$ is defined as the difference of the energy of the dimer $E_\mathrm{opt}[\ce{(XY3)2}]$ and double the energy of the monomer $E_\mathrm{opt}[\ce{XY3}]$ at their optimised (relaxed) geometries $\eqref{e-diss-def}$.
The interaction energy is defined as the difference of energy of the relaxed dimer and double the energy of the monomers in the geometry of the dimer $E_\mathrm{frag}[\ce{(XY3)^{\neq}}]$ $\eqref{e-int-def}$. That basically means breaking the molecule in two parts, but keeping these fragments in the same geometry.
The deformation energy (or preparation energy) is defined as the difference of the energy of the optimised and the non-optimised monomer $\eqref{e-def-def}$. This is the energy required to distort the monomer (in its ground state) to the configuration it will have in the dimer.
$$\begin{align}
E_\mathrm{diss} &=
  E_\mathrm{opt}[\ce{(XY3)2}] - 2E_\mathrm{opt}[\ce{XY3}]
  \tag1\label{e-diss-def}\\
E_\mathrm{int} &=
  E_\mathrm{opt}[\ce{(XY3)2}] - 2E_\mathrm{frag}[\ce{(XY3)^{\neq}}]
  %\ddag not implemented
  \tag2\label{e-int-def}\\
E_\mathrm{def} &=
  E_\mathrm{frag}[\ce{(XY3)^{\neq}}] - E_\mathrm{opt}[\ce{XY3}]
  \tag3\label{e-def-def}\\
E_\mathrm{diss} &=
  E_\mathrm{int} + 2E_\mathrm{def}\tag{1'}
\end{align}$$
Results & Discussion
The Monomers $\ce{XCl3; X{=}\{B,Al\}}$.
Let's just get the obvious out of the way: Boron is (vdW-radius 205 pm) smaller than aluminium (vdW-radius 240 pm). For comparison chlorine has a vdW-radius of 205 pm, too. That is pretty much reflected in the bond lengths and the chlorine-chlorine distance.
\begin{array}{llrrr}\hline
&\ce{X{=}}& \ce{Al} &\ce{B} &\ce{Cl}\\\hline
\mathbf{d}(\ce{X-Cl})&/\pu{pm} & 206.0 &173.6&--\\
\mathbf{d}(\ce{Cl\bond{~}Cl'})&/\pu{pm} & 356.8 & 300.6 & --\\\hline
\mathbf{r}_\mathrm{vdW}&/\pu{pm} & 240   & 205   & 205\\
\mathbf{r}_\mathrm{sing}&/\pu{pm} & 126   & 85   & 99\\
\mathbf{r}_\mathrm{doub}&/\pu{pm} & 113   & 78   & 95\\\hline
\end{array}
From this data we can draw certain conclusions without further looking. The boron monomer is much more compact than the aluminium monomer. When we compare the bond lengths to the covalent radii (Pyykkö and Atsumi) we find that the boron chloride bond is about the length that we would expect from a double bond ($\mathbf{r}_\mathrm{doub}(\ce{B}) + \mathbf{r}_\mathrm{doub}(\ce{Cl}) = 173~\pu{pm}$). While the aluminium chloride bond is still significantly shorter than a single bond ($\mathbf{r}_\mathrm{sing}(\ce{Al}) + \mathbf{r}_\mathrm{sing}(\ce{Cl}) = 225~\pu{pm}$), it is still also much longer than a double bond ($\mathbf{r}_\mathrm{doub}(\ce{Al}) + \mathbf{r}_\mathrm{doub}(\ce{Cl}) = 191~\pu{pm}$).
This itself offers compelling evidence, that there is more π-backbonding in $\ce{BCl3}$ than in $\ce{AlCl3}$. Molecular orbital theory offers more evidence for this. In both compounds is a doubly occupied π orbital. The following pictures are for a contour value of 0.05; aluminium (left/top) and boron (right/bottom)

In numbers, the main contributions are as follows (this is just a representation, not the actual formula):
$$\begin{align}
\pi(\ce{BCl3}) &=
   21\%~\ce{p_{$z$}-B} + \sum_{i=1}^3 26\%~\ce{p_{$z$}-Cl^{$(i)$}}\\
\pi(\ce{AlCl3}) &=
   13\%~\ce{p_{$z$}-Al} + \sum_{i=1}^3 29\%~\ce{p_{$z$}-Cl^{$(i)$}}
\end{align}$$
There is still some more evidence. The natural atomic charges (NPA of NBO6) fairly well agree with that assesment; aluminium is far more positive than boron.
$$\begin{array}{lrr}
                    & \ce{AlCl3} & \ce{BCl3}\\\hline
\mathbf{q}(\ce{X})~\text{[NPA]}  & +1.4       & +0.3     \\
\mathbf{q}(\ce{Cl})~\text{[NPA]} & -0.5       & -0.1     \\\hline
%\mathbf{q}(\ce{X})~\text{[QTAIM]}  & +2.4       & +2.0    \\
%\mathbf{q}(\ce{Cl})~\text{[QTAIM]} & -0.8       & -0.7     \\\hline
\end{array}$$
The analysis in terms of QTAIM also shows that the bonds in $\ce{AlCl3}$ they are predominantly ionic (left/top) while in $\ce{BCl3}$ are predominantly covalent (right/bottom).

One final thought on the bonding can be supplied with a natural resonance theory analysis (NBO6). I have chosen the following starting configurations and let the program calculate their contribution.

The overall structures in terms of resonance are the same for both cases, that is if you force resonance treatment of the aluminium monomer. Structure A does not contribute, while the others contribute to about 31%. However, when not forced into resonance, structure A is the best approximation of the bonding situation for $\ce{AlCl3}$. In the case of $\ce{BCl3}$ the algorithm finds a hyperbond between the chlorine atoms, a strongly delocalised bond between multiple centres. In this case these are 3-centre-4-electron bonds between the chlorine atoms, resulting from the higher lying degenerated π orbitals.

This all is quite good evidence that the monomer of boron chloride should be more stable towards dimerisation than the monomer of aluminium.
The Dimers $\ce{(XCl3)2; X{=}\{B,Al\}}$.
The obvious change is that the co-ordination of the central elements goes from trigonal planar to distorted tertrahedral. A look at the geometries will give us something to talk about.
\begin{array}{llrrr}\hline
&\ce{X{=}}& \ce{Al} &\ce{B} &\ce{Cl}\\\hline
\mathbf{d}(\ce{X-Cl})&/\pu{pm} & 206.7 &175.9&--\\
\mathbf{d}(\ce{X-{\mu}Cl})&/\pu{pm} & 226.1 &198.7&--\\
\mathbf{d}(\ce{Cl\bond{~}{\mu}Cl})&/\pu{pm} & 354.1 & 308.0 & --\\
\mathbf{d}(\ce{{\mu}Cl\bond{~}{\mu}Cl'})&/\pu{pm} & 323.6 & 287.3 & --\\
\mathbf{d}(\ce{B\bond{~}B'})&/\pu{pm} & 315.7 & 274.7 & --\\\hline
\mathbf{r}_\mathrm{vdW}&/\pu{pm} & 240   & 205   & 205\\
\mathbf{r}_\mathrm{sing}&/\pu{pm} & 126   & 85   & 99\\
\mathbf{r}_\mathrm{doub}&/\pu{pm} & 113   & 78   & 95\\\hline
\end{array}
In principle nothing much changes other than the expected elongation of the bonds that are now bridging. In case of aluminium the stretch is just below 10% and for boron it is slightly above 14%, having a bit more impact. In the boron dimer also the terminal bonds are slightly (> +1%) affected, while for aluminium there is almost no change.
The charges are not really a reliable tool, especially when they are that close to zero as they are for boron. In both cases one can see that charge density is transferred from the bridging chlorine to the central $\ce{X}$.
$$\begin{array}{lrr}
                    & \ce{(AlCl3)2} & \ce{(BCl3)2}\\\hline
\mathbf{q}(\ce{X})~\text{[NPA]}  & +1.3       & +0.2     \\
\mathbf{q}(\ce{Cl})~\text{[NPA]} & -0.5       & -0.1     \\\hline
\mathbf{q}(\ce{{\mu}Cl})~\text{[NPA]} & -0.4       & +0.1     \\\hline
\end{array}$$
A look at the central four-membered-ring of in terms of QTAIM offers that the overall bonding does not change. In aluminium they get a little more ionic, while in boron they stay largely covalent.

The NBO analysis offers a maybe quite surprising result. There are no hyperbonds in any of the dimers. While a description in these terms is certainly possible, after all it is just an interpretation tool, it is completely unnecessary. So after all we have two kinds of bonds in the dimers four terminal $\ce{X-Cl}$ and four bridging $\ce{X-{\mu}Cl}$ bonds. Therefore the most accurate description is with formal charges (also the simplest). The notation with the arrows is not wrong, but it does not represent the fact that the bonds are equal for symmetry reasons alone.

To make this straight: There are no hyperbonds in $\ce{(XCl3)2; X{=}\{B,Al\}}$; this includes three-centre-two-electron bonds, and three-centre-four-electron bonds. And deeper insight to those will be offered on another day.
The differentiation between a dative bond and some other for of bond does not make sense, as the bonds are equal and only introduced by a deficiency of the used description model.
A natural resonance theory for $\ce{(BCl3)2}$ gives us a overall contribution of the main (all single bonds) structure of 46%; while all other structure do contribute, there are too many and their contribution is too little (> 5%). I did not run this analysis for the aluminium case as I did not expect any more insight and I did not want to waste calculation time.
Dimerisation - yes or no
The energies offer us a clear trend. Aluminium likes to dimerise, boron not. However, there are still some things to discuss. I am going to argue for the reaction
$$\begin{align}
\ce{2XCl3 &-> (XCl3)2}&
\Delta E-\mathrm{diss}/E_\mathrm{o}/H/G&.
\end{align};$$
therefore if reaction energies are negative the dimerisation is favoured.
The following table includes all calculated energies, including the energy decomposition analysis mentioned at the beginning. All energies are given in $\pu{kJ mol^-1}$.
\begin{array}{lrcrcrcrr}
 \Delta & E_\mathrm{diss} &(& E_\mathrm{int} &+2\times& E_\mathrm{def}&)& E_\mathrm{o} &H &G\\\hline
\ce{Al} & -113.5 &(& -224.2 &+2\times& 55.4&)& -114.7 & -60.4 & -230.4\\
\ce{B}  &   76.4 &(& -111.2 &+2\times& 93.8&)&   82.6 & -47.1 &  152.5\\\hline
\end{array}
The result is fairly obvious at first. The association for aluminium is strongly exergonic, while for boron it is strongly endergonic. While both reactions should be exothermic, stronger for aluminium, the trend for the observed electronic energies ($E_\mathrm{o}$ including the zero-point energy correction) and the (electronic) dissociation energies reflect the overall trend for the Gibbs enthalpies.
While it is fairly surprising how strongly entropy favours association of $\ce{AlCl3}$, it is also surprising how it strongly disfavours it for $\ce{BCl3}$.
A look at the decomposed electronic energy offers great insight into the reasons why one dimer is stable and the other not (at room temperature).
The interaction energy of the fragments is double for aluminium than it is for boron. This can be traced back to the very large difference in the atomic partial charges. One could expect that the electrostatic energy is a lot more attractive for aluminium than it is for boron.
The deformation energy on the other hand clearly reflects the changes in the geometry discussed above. For aluminium there is a smaller penalty resulting from the elongation of the $\ce{Al-Cl}$ bond and pyramidalisation. For boron on the other hand this has a 1.5 times larger effect. The distortion also weakens the π-backbonding, which the additional bonding would need to compensate.
The four-membered-ring is certainly not an ideal geometry and the bridging chlorine atoms come dangerously close.
Conclusion, Summary and TL;DR:
The distortion of the geometry of the monomer $\ce{BCl3}$ cannot be compensated by the additional bonding between the two fragments. Therefore the monomers are more stable than the dimer. Additionally entropy considerations at room temperature favour the monomer, too.
On the other hand, the distortion of the molecular geometry in $\ce{AlCl3}$ is less severe. The gain in interaction energy of the two fragments well overcompensates for the change. Entropy also favours the dimerisation.
While size of the central atom is certainly the distinguishing factor, its impact is only severe on  the electronic structure. Steric crowding would not be a problem when the interaction energy would compensate for that. This is quite evident because $\ce{BCl3}$ is still a very good Lewis acid and forms stable compounds with much larger moieties than itself.
References
The used van der Waals radii were taken from S. S. Batsanov Inorg. Mat. 2001, 37 (9), 871-885. And the covalent radii have been taken from P. Pyykkö and M. Atsumi Chem. Eur. J. 2009, 15, 12770-12779.
Computations have been carried out using Gaussian 09 rev D.01 with NBO 6.0. Additional analyses have been performed with MultiWFN 3.3.8. Orbital pictures were generated with the incredible ChemCraft.


Question:
My understanding is that light can not escape from within a black hole (within the event horizon). I've also heard that information cannot propagate faster than the speed of light. I assume that the gravitational attraction caused by a black hole carries information about the amount of mass within the black hole.
So, how does this information escape?
Looking at it from a particle point of view: do the gravitons (should they exist) travel faster than the photons?

There are some good answers here already but I hope this is a nice short summary:
Electromagnetic radiation cannot escape a black hole, because it travels at the speed of light. Similarly, gravitational radiation cannot escape a black hole either, because it too travels at the speed of light. If gravitational radiation could escape, you could theoretically use it to send a signal from the inside of the black hole to the outside, which is forbidden.
A black hole, however, can have an electric charge, which means there is an electric field around it. This is not a paradox because a static electric field is different from electromagnetic radiation. Similarly, a black hole has a mass, so it has a gravitational field around it. This is not a paradox either because a gravitational field is different from gravitational radiation.
You say the gravitational field carries information about the amount of mass (actually energy) inside, but that does not give a way for someone inside to send a signal to the outside, because to do so they would have to create or destroy energy, which is impossible. Thus there is no paradox.


Question:
I've heard that the Hilbert transform can be used to calculate the envelope of a signal.  How does this work? And how is this "Hilbert envelope" different from the envelope one gets by simply rectifying a signal?
I'm interested specifically in finding a way to calculate an envelope for use in dynamic range compression (i.e., "turning down the volume" of the loud parts of an audio signal automatically).

The Hilbert transform is used to calculate the "analytic" signal. See for example  If your signal is a sine wave or an modulated sine wave, the magnitude of the analytic signal will indeed look like the envelope. However, the computation of the Hilbert transform is not trivial. Technically it requires a non-causal FIR filter of considerable length so it will require a fair amount of MIPS, memory and latency.
For a broad band signal, it really depends on how you define "envelope" for your specific application. For your application of dynamic range compression you want a metric that is well correlated with the the perception of loudness over time. The Hilbert Transform is not the right tool for that.
A better option would be to apply an A-weighted filter ( and then do a lossy peak or lossy RMS detector. This will correlate fairly well with perceived loudness over time and is relatively cheap to do.


Question:
I know that the cells of mammals at least stop dividing when they are old, and then die a programmed cell death. Then other cells have to replace them.
But in a bacterial colony, each cell replicates for itself. Obviously, if a division of a bacterial cell of generation N were to produce two new cells of generation N+1, and all bacteria died of old age at generation M, there would be no bacteria left around.
So how is it regulated in bacteria? Are their divisions simply unlimited? Does a cell never die and just divide forever?

This is a interesting question and for a long time it was thought that they do not age. In the meantime there are some new papers which say that bacteria do indeed age.
Aging can be defined as the accumulation of non-genetic damages (for example oxidative damage to proteins) over time. If too much of these damages are accumulated, the cell will eventually die.
For bacteria there seems to be an interesting way around this. The second paper cited below found that bacteria do not divide symmetrically into two daughter cells, but seem to split into one cell which receives more damage and one which receives less. The latter one can be called rejuvenated and seems to make sure that the bacteria can seemingly divide forever. Using this strategy limits the non-genetic damage to relatively few cells (if you consider the doubling mechanism) which could eventually die to save the others.
Have a look at the following publications which go into detail (the first is a summary of the second but worth reading):

Do bacteria age? Biologists discover the answer follows simple
economics
Temporal Dynamics of Bacterial Aging and Rejuvenation
Aging and death in an organism that reproduces by morphologically
symmetric division.


Question: If I write on the starting page of a notebook, it will write well. But when there are few or no pages below the page where I am writing, the pen will not write well. Why does this happen?

I'd say the culprit is the contact area between the two surfaces relative to the deformation.
When there are other pieces of paper below it, all the paper is able to deform when you push down; because the paper is fairly soft and deformable fiber.  If there is more soft deformable paper below it, the layers are able to bend and stretch more.
(A simplified example of this is Springs in series, where the overall stiffness decreases when you stack up multiple deformable bodies in a row)
This deformation creates the little indents on the page (and on pages below it; you can often see on the next page the indents for the words you wrote on the page above).  The deeper these indents are, the more of the ballpoint is able to make contact with the surface.

If there is barely any deformation, then the flat surface doesn't get to make good contact with the page.  This makes it hard for the tip of the pen to actually roll, which is what moves the ink from the cartridge to the tip.  It would also make a thinner line due to less contact area.
Here is an amazing exaggerated illustration I made on Microsoft Paint:

The top one has more pages, the bottom one has fewer.  I've exaggerated how much the pages deform obviously; but the idea is that having more pages below with make that indent larger; leading to the increased surface area on the pen tip.
Note that this doesn't really apply to other types of pens.  Pens that use other ways to get the ink out have less of an issue writing with solid surfaces behind; but ballpoint pens are usually less expensive and more common.


Question:
I know how to code for factorials using both iterative and recursive (e.g. n * factorial(n-1) for e.g.). I read in a textbook (without been given any further explanations) that there is an even more efficient way of coding for factorials by dividing them in half recursively.
I understand why that may be the case. However I wanted to try coding it on my own, and I don't think I know where to start though. A friend suggested I write base cases first. and I was thinking of using arrays so that I can keep track of the numbers... but I really can't see any way out to designing such a code.
What kind of techniques should I be researching?

The best algorithm that is known is to express the factorial as a product of prime powers.  One can quickly determine the primes as well as the right power for each prime using a sieve approach.  Computing each power can be done efficiently using repeated squaring, and then the factors are multiplied together.  This was described by Peter B. Borwein, On the Complexity of Calculating Factorials, Journal of Algorithms 6 376–380, 1985. (PDF)  In short, $n!$ can be computed in $O(n(\log n)^3\log \log n)$ time, compared to the $\Omega(n^2 \log n)$ time required when using the definition.
What the textbook perhaps meant was the divide-and-conquer method.  One can reduce the $n-1$ multiplications by using the regular pattern of the product.
Let $n?$ denote $1 \cdot 3 \cdot 5 \dotsm (2n-1)$ as a convenient notation.
Rearrange the factors of $(2n)! = 1 \cdot 2 \cdot 3 \dotsm (2n)$ as
$$(2n)! = n! \cdot 2^n \cdot 3 \cdot 5 \cdot 7 \dotsm (2n-1).$$
Now suppose $n = 2^k$ for some integer $k>0$.
(This is a useful assumption to avoid complications in the following discussion, and the idea can be extended to general $n$.)
Then $(2^k)! = (2^{k-1})!2^{2^{k-1}}(2^{k-1})?$ and by expanding this recurrence,
$$(2^k)! =  \left(2^{2^{k-1}+2^{k-2}+\dots+2^0}\right) \prod_{i=0}^{k-1} (2^i)? = \left(2^{2^k - 1}\right) \prod_{i=1}^{k-1} (2^i)?.$$
Computing $(2^{k-1})?$ and multiplying the partial products at each stage takes $(k-2) + 2^{k-1} - 2$ multiplications.  This is an improvement of a factor of nearly $2$ from $2^k-2$ multiplications just using the definition.  Some additional operations are required to compute the power of $2$, but in binary arithmetic this can be done cheaply (depending on what precisely is required, it may just require adding a suffix of $2^k-1$ zeroes).
The following Ruby code implements a simplified version of this.  This does not avoid recomputing $n?$ even where it could do so:
def oddprod(l,h)
  p = 1
  ml = (l%2>0) ? l : (l+1)
  mh = (h%2>0) ? h : (h-1)
  while ml <= mh do
    p = p * ml
    ml = ml + 2
  end
  p
end

def fact(k)
  f = 1
  for i in 1..k-1
    f *= oddprod(3, 2 ** (i + 1) - 1)
  end
  2 ** (2 ** k - 1) * f
end

print fact(15)

Even this first-pass code improves on the trivial
f = 1; (1..32768).map{ |i| f *= i }; print f

by about 20% in my testing.
With a bit of work, this can be improved further, also removing the requirement that $n$ be a power of $2$ (see the extensive discussion).


Question: In my linear algebra class, we just talked about determinants. So far I’ve been understanding the material okay, but now I’m very confused. I get that when the determinant is zero, the matrix doesn’t have an inverse. I can find the determinant of a $2\times 2$ matrix by the formula. Our teacher showed us how to compute the determinant of an $n \times n$ matrix by breaking it up into the determinants of smaller matrices. Apparently there is a way by summing over a bunch of permutations. But the notation is really hard for me and I don’t really know what’s going on with them anymore. Can someone help me figure out what a determinant is, intuitively, and how all those definitions of it are related?

Your trouble with determinants is pretty common. They’re a hard thing to teach well, too, for two main reasons that I can see: the formulas you learn for computing them are messy and complicated, and there’s no “natural” way to interpret the value of the determinant, the way it’s easy to interpret the derivatives you do in calculus at first as the slope of the tangent line. It’s hard to believe things like the invertibility condition you’ve stated when it’s not even clear what the numbers mean and where they come from.
Rather than show that the many usual definitions are all the same by comparing them to each other, I’m going to state some general properties of the determinant that I claim are enough to specify uniquely what number you should get when you put in a given matrix. Then it’s not too bad to check that all of the definitions for determinant that you’ve seen satisfy those properties I’ll state.
The first thing to think about if you want an “abstract” definition of the determinant to unify all those others is that it’s not an array of numbers with bars on the side. What we’re really looking for is a function that takes N vectors (the N columns of the matrix) and returns a number. Let’s assume we’re working with real numbers for now.
Remember how those operations you mentioned change the value of the determinant?

Switching two rows or columns changes the sign.

Multiplying one row by a constant multiplies the whole determinant by that constant.

The general fact that number two draws from: the determinant is linear in each row. That is, if you think of it as a function $\det: \mathbb{R}^{n^2} \rightarrow \mathbb{R}$, then $$ \det(a \vec v_1 +b \vec w_1 , \vec v_2 ,\ldots,\vec v_n ) = a \det(\vec v_1,\vec v_2,\ldots,\vec v_n) + b \det(\vec w_1, \vec v_2, \ldots,\vec v_n),$$ and the corresponding condition in each other slot.

The determinant of the identity matrix $I$ is $1$.

I claim that these facts are enough to define a unique function that takes in N vectors (each of length N) and returns a real number, the determinant of the matrix given by those vectors. I won’t prove that, but I’ll show you how it helps with some other interpretations of the determinant.
In particular, there’s a nice geometric way to think of a determinant. Consider the unit cube in N dimensional space: the set of N vectors of length 1 with coordinates 0 or 1 in each spot. The determinant of the linear transformation (matrix) T is the signed volume of the region gotten by applying T to the unit cube. (Don’t worry too much if you don’t know what the “signed” part means, for now).
How does that follow from our abstract definition?
Well, if you apply the identity to the unit cube, you get back the unit cube. And the volume of the unit cube is 1.
If you stretch the cube by a constant factor in one direction only, the new volume is that constant. And if you stack two blocks together aligned on the same direction, their combined volume is the sum of their volumes: this all shows that the signed volume we have is linear in each coordinate when considered as a function of the input vectors.
Finally, when you switch two of the vectors that define the unit cube, you flip the orientation. (Again, this is something to come back to later if you don’t know what that means).
So there are ways to think about the determinant that aren’t symbol-pushing. If you’ve studied multivariable calculus, you could think about, with this geometric definition of determinant, why determinants (the Jacobian) pop up when we change coordinates doing integration. Hint: a derivative is a linear approximation of the associated function, and consider a “differential volume element” in your starting coordinate system.
It’s not too much work to check that the area of the parallelogram formed by vectors $(a,b)$ and $(c,d)$ is $\Big|{}^{a\;b}_{c\;d}\Big|$
either: you might try that to get a sense for things.


Question: What are the definitions of these three things and how are they related? I've tried looking online but there is no concrete answer online for this question.

Here's a graphic I use to explain the difference in my general chemistry courses:

All electrons that have the same value for $n$ (the principle quantum number) are in the same shell
Within a shell (same $n$), all electrons that share the same $l$ (the angular momentum quantum number, or orbital shape) are in the same sub-shell
When electrons share the same $n$, $l$, and $m_l$, we say they are in the same orbital (they have the same energy level, shape, and orientation)

So to summarize:

same $n$ - shell
same $n$ and $l$ - sub-shell
same $n$, $l$, and $m_l$ - orbital

Now, in the other answer, there is some discussion about spin-orbitals, meaning that each electron would exist in its own orbital. For practical purposes, you don't need to worry about that - by the time those sorts of distinctions matter to you, there won't be any confusion about what people mean by "shells" and "sub-shells." For you, for now, orbital means "place where up to two electrons can exist," and they will both share the same $n$, $l$, and $m_l$ values, but have opposite spins ($m_s$).


Question:
I am used to thinking of finite-differences as a special case of finite-elements, on a very constrained grid. So what are the conditions on how to choose between Finite Difference Method (FDM) and Finite Element Method (FEM) as a numerical method?
On the side of Finite Difference Method (FDM), one may count that they are conceptually simpler and easier to implement than Finite Element Method (FEM). FEM have the benefit of being very flexible, e.g., the grids may be very non-uniform and the domains may have arbitrary shape.
The only example I know where FDM has turned out superior to FEM is in
Celia, Bouloutas, Zarba, where the benefit is due to the FD method using a different discretization of time derivative, which, however, could be fixed for the finite element method.

It is possible to write most specific finite difference methods as Petrov-Galerkin finite element methods with some choice of local reconstruction and quadrature, and most finite element methods can also be shown to be algebraically equivalent to some finite difference method. Therefore, we should choose a method based on which analysis framework we want to use, which terminology we like, which system for extensibility we like, and how we would like to structure software. The following generalizations hold true in the vast majority of variations in practical use, but many points can be circumvented.
Finite Difference
Pros

efficient quadrature-free implementation
aspect ratio independence and local conservation for certain schemes (e.g. MAC for incompressible flow)
robust nonlinear methods for transport (e.g. ENO/WENO)
M-matrix for some problems
discrete maximum principle for some problems (e.g. mimetic finite differences)
diagonal (usually identity) mass matrix
inexpensive nodal residual permits efficient nonlinear multigrid (FAS)
cell-wise Vanka smoothers give efficient matrix-free smoothers for incompressible flow

Cons

more difficult to implement "physics"
staggered grids are sometimes quite technical
higher than second order on unstructured grids is difficult
no Galerkin orthogonality, so convergence may be more difficult to prove
not a Galerkin method, so discretization and adjoints do not commute (relevant to optimization and inverse problems)
self-adjoint continuum problems often yield non-symmetric matrices
solution is only defined pointwise, so reconstruction at arbitrary locations is not uniquely defined
boundary conditions tend to be complicated to implement
discontinuous coefficients usually make the methods first order
stencil grows if physics includes "cross terms"

Finite Element
Pros

Galerkin orthogonality (discrete solution to coercive problems is within a constant of the best solution in the space)
simple geometric flexibility
discontinuous Galerkin offers robust transport algorithm, arbitrary order on unstructured grids
cellwise entropy inequality guaranteeing $L^2$ stability holds independent of mesh, dimension, order of accuracy, and presence of discontinuous solutions, without needing nonlinear limiters
easy of implementing boundary conditions
can choose conservation statement by choosing test space
discretization and adjoints commute (for Galerkin methods)
elegant foundation in functional analysis
at high order, local kernels can exploit tensor product structure that is missing with FD
Lobatto quadrature can make methods energy-conserving (assuming a symplectic time integrator)
high order accuracy even with discontinuous coefficients, as long as you can align to boundaries
discontinuous coefficients inside elements can be accommodated with XFEM
easy to handle multiple inf-sup conditions

Cons

many elements have trouble at high aspect ratio
continuous FEM has trouble with transport (SUPG is diffusive and oscillatory)
DG usually has more degrees of freedom for same accuracy (though HDG is much better)
continuous FEM does not provide cheap nodal problems, so nonlinear smoothers have much poorer constants
usually more nonzeros in assembled matrices
have to choose between consistent mass matrix (some nice properties, but has full inverse, thus requiring an implicit solve per time step) and lumped mass matrix.


Question:
As part of some blockchain-related research I am currently undertaking, the notion of using blockchains for a variety of real-world applications are thrown about loosely.
Therefore, I propose the following questions:

What important/crucial real-world applications use blockchain?
To add on to the first question, more specifically, what real-world applications actually need blockchain - who may or may not currently use it?

From a comment, I further note that this disregards the notion of cryptocurrencies. However, the use of smart contracts can have other potential applications aside from benefits they can pose to the area of cryptocurrencies

Apart from Bitcoin and Ethereum (if we are generous) there are no major and
important uses today.
It is important to notice that blockchains have some severe limitations.  A
couple of them being:

It only really works for purely digital assets
The digital asset under control needs to keep its value even if it's public
All transactions need to be public
A rather bad confirmation time
Smart contracts are scary

Purely digital assets
If an asset is actually a physical asset with just a digital "twin" that is
being traded, we will risk that local jurisdiction (i.e. your law enforcement)
can have a different opinion of ownership than what is on the blockchain.
To take an example; suppose that we are trading (real and physical) bikes on the
blockchain, and that on the blockchain, we put its serial number.  Suppose
further that I hack your computer and put the ownership of your bike to be me.
Now, if you go to the police, you might be able to convince them that the real
owner of the bike is you, and thus I have to give it back.  However, there is no
way of making me give you the digital twin back, thus there is a dissonance: the
bike is owned by you, but the blockchain claims it's owned by me.
There are many such proposed use cases (trading physical goods on a blockchain)
out in the open of trading bikes, diamonds, and even oil.
The digital assets keep value even if public
There are many examples where people want to put assets on the blockchain, but
are somehow under the impression that that gives some kind of control.  For
instance, musician Imogen Heap is creating a product in which all musicians
should put their music on the blockchain and automatically be paid when a radio
plays your hit song.  They are under the impression that this creates an
automatic link between playing the song and paying for the song.
The only thing it really does is to create a very large database for music which
is probably quite easy to download.
There is currently no way around having to put the full asset visible on the
chain.  Some people are talking about "encryptions", "storing only the hash",
etc., but in the end, it all comes down to: publish the asset, or don't
participate.
Public transactions
In business it is often important to keep your cards close to your chest.  You
don't want real time exposure of your daily operations.
Some people try to make solutions where we put all the dairy farmers' production
on the blockchain together with all the dairy stores' inventory.  In this way we
can easily send trucks to the correct places!  However, this makes both farmers
and traders liable for inflated prices if they are overproducing/under-stocked.
Other people want to put energy production (solar panels, wind farms) on the
blockchain.  However, no serious energy producer will have real time production
data out for the public.  This has major impact on the stock value and that kind
of information is the type you want to keep close to your chest.
This also holds for so-called green certificates, where you ensure you only
use "green energy".
Note:  There are theoretical solutions that build on zero-knowledge proofs
that would allow transactions to be secret.  However, these are nowhere near
practical yet, and time will show if this item can be fixed.
Confirmation time
You can, like Ethereum, make the block time as small as you would like.  In
Bitcoin, the block time is 10 minutes, and in Ethereum it is
less than a minute (I don't remember the specific figure).
However, the smaller block time, the higher the chance of long-lived forks.  To
ensure your transaction is confirmed you still have to wait quite long.
There are currently no good solutions here either.
Smart contracts are scary
Smart contract are difficult to write.  They are computer programs that move
assets from one account to another (or more complicated).  However, we want
traders and "normal" people to be able to write these contracts, and not rely on
computer science programming experts.  You can't undo a transaction.  This is a
tough nut to crack!
If you are doing high value trading, and end up writing a zero too much in the
transaction (say \$10M instead of \$1M), you call your bank immediately!  That
fixes it.  If not, let's hope you have insurance.  In a blockchain setting, you
have neither a bank, nor insurance.  Those \$9M are gone and it was due to a
typo in a smart contract or in a transaction.
Smart contracts is really playing with fire.  It's too easy to empty all your
assets in a single click.  And it has happened, several times. People have lost hundreds of millions of dollars due to smart contract errors.
Source: I am working for an energy company doing wind and solar energy
production as well as trading oil and gas.  Have been working on blockchain
solution projects.


Question:
I haven't seen the question stated precisely in these terms, and this is why I make a new question.
What I am interested in knowing is not the definition of a neural network, but understanding the actual difference with a deep neural network.
For more context: I know what a neural network is and how backpropagation works. I know that a DNN must have multiple hidden layers. However, 10 years ago in class I learned that having several layers or one layer (not counting the input and output layers) was equivalent in terms of the functions a neural network is able to represent (see Cybenko's Universal approximation theorem), and that having more layers made it more complex to analyse without gain in performance. Obviously, that is not the case anymore.
I suppose, maybe wrongly, that the differences are in terms of training algorithm and properties rather than structure, and therefore I would really appreciate if the answer could underline the reasons that made the move to DNN possible (e.g. mathematical proof or randomly playing with networks?) and desirable (e.g. speed of convergence?)

Let's start with a triviliaty: Deep neural network is simply a feedforward network with many hidden layers.
This is more or less all there is to say about the definition. Neural networks can be recurrent or feedforward; feedforward ones do not have any loops in their graph and can be organized in layers. If there are "many" layers, then we say that the network is deep.
How many layers does a network have to have in order to qualify as deep? There is no definite answer to this  (it's a bit like asking how many grains make a heap), but usually having two or more hidden layers counts as deep. In contrast, a network with only a single hidden layer is conventionally called "shallow". I suspect that there will be some inflation going on here, and in ten years people might think that anything with less than, say, ten layers is shallow and suitable only for kindergarten exercises. Informally, "deep" suggests that the network is tough to handle.
Here is an illustration, adapted from here:

But the real question you are asking is, of course, Why would having many layers be beneficial?
I think that the somewhat astonishing answer is that nobody really knows. There are some common explanations that I will briefly review below, but none of them has been convincingly demonstrated to be true, and one cannot even be sure that having many layers is really beneficial.
I say that this is astonishing, because deep learning is massively popular, is breaking all the records (from image recognition, to playing Go, to automatic translation, etc.) every year, is getting used by the industry, etc. etc. And we are still not quite sure why it works so well.
I base my discussion on the Deep Learning book by Goodfellow, Bengio, and Courville which went out in 2017 and is widely considered to be the book on deep learning. (It's freely available online.) The relevant section is 6.4.1 Universal Approximation Properties and Depth.
You wrote that

10 years ago in class I learned that having several layers or one layer (not counting the input and output layers) was equivalent in terms of the functions a neural network is able to represent [...]

You must be referring to the so called Universal approximation theorem, proved by Cybenko in 1989 and generalized by various people in the 1990s. It basically says that a shallow neural network (with 1 hidden layer) can approximate any function, i.e. can in principle learn anything. This is true for various nonlinear activation functions, including rectified linear units that most neural networks are using today (the textbook references Leshno et al. 1993 for this result).
If so, then why is everybody using deep nets?
Well, a naive answer is that because they work better. Here is a figure from the Deep Learning book showing that it helps to have more layers in one particular task, but the same phenomenon is often observed across various tasks and domains:

We know that a shallow network could perform as good as the deeper ones. But it does not; and they usually do not. The question is --- why? Possible answers:

Maybe a shallow network would need more neurons then the deep one?
Maybe a shallow network is more difficult to train with our current algorithms (e.g. it has more nasty local minima, or the convergence rate is slower, or whatever)?
Maybe a shallow architecture does not fit to the kind of problems we are usually trying to solve (e.g. object recognition is a quintessential "deep", hierarchical process)?
Something else?

The Deep Learning book argues for bullet points #1 and #3. First, it argues that the number of units in a shallow network grows exponentially with task complexity. So in order to be useful a shallow network might need to be very big; possibly much bigger than a deep network. This is based on a number of papers proving that shallow networks would in some cases need exponentially many neurons; but whether e.g. MNIST classification or Go playing are such cases is not really clear. Second, the book says this:

Choosing a deep model encodes a very general belief that the function we
  want to learn should involve composition of several simpler functions. This can be
  interpreted from a representation learning point of view as saying that we believe
  the learning problem consists of discovering a set of underlying factors of variation
  that can in turn be described in terms of other, simpler underlying factors of
  variation.

I think the current "consensus" is that it's a combination of bullet points #1 and #3: for real-world tasks deep architecture are often beneficial and shallow architecture would be inefficient and require a lot more neurons for the same performance.
But it's far from proven. Consider e.g. Zagoruyko and  Komodakis, 2016, Wide Residual Networks. Residual networks with 150+ layers appeared in 2015 and won various image recognition contests. This was a big success and looked like a compelling argument in favour of deepness; here is one figure from a presentation by the first author on the residual network paper (note that the time confusingly goes to the left here):

But the paper linked above shows that a "wide" residual network with "only" 16 layers can outperform "deep" ones with 150+ layers. If this is true, then the whole point of the above figure breaks down.
Or consider Ba and Caruana, 2014, Do Deep Nets Really Need to be Deep?:

In this paper we provide empirical evidence that shallow nets are capable of learning the same
  function as deep nets, and in some cases with the same number of parameters as the deep nets. We
  do this by first training a state-of-the-art deep model, and then training a shallow model to mimic the
  deep model. The mimic model is trained using the model compression scheme described in the next
  section. Remarkably, with model compression we are able to train shallow nets to be as accurate
  as some deep models, even though we are not able to train these shallow nets to be as accurate as
  the deep nets when the shallow nets are trained directly on the original labeled training data. If a
  shallow net with the same number of parameters as a deep net can learn to mimic a deep net with
  high fidelity, then it is clear that the function learned by that deep net does not really have to be deep.

If true, this would mean that the correct explanation is rather my bullet #2, and not #1 or #3.
As I said --- nobody really knows for sure yet.

Concluding remarks
The amount of progress achieved in the deep learning over the last ~10 years is truly amazing, but most of this progress was achieved by trial and error, and we still lack very basic understanding about what exactly makes deep nets to work so well. Even the list of things that people consider to be crucial for setting up an effective deep network seems to change every couple of years.
The deep learning renaissance started in 2006 when Geoffrey Hinton (who had been working on neural networks for 20+ years without much interest from anybody) published a couple of breakthrough papers offering an effective way to train deep networks (Science paper, Neural computation paper). The trick was to use unsupervised pre-training before starting the gradient descent. These papers revolutionized the field, and for a couple of years people thought that unsupervised pre-training was the key.
Then in 2010 Martens showed that deep neural networks can be trained with second-order methods (so called Hessian-free methods) and can outperform networks trained with pre-training: Deep learning via Hessian-free optimization. Then in 2013 Sutskever et al. showed that  stochastic gradient descent with some very clever tricks can outperform  Hessian-free methods: On the importance of initialization and momentum in deep learning. Also, around 2010 people realized that using rectified linear units instead of sigmoid units makes a huge difference for gradient descent. Dropout appeared in 2014. Residual networks appeared in 2015. People keep coming up with more and more effective ways to train deep networks and what seemed like a key insight 10 years ago is often considered a nuisance today. All of that is largely driven by trial and error and there is little understanding of what makes some things work so well and some other things not. Training deep networks is like a big bag of tricks. Successful tricks are usually rationalized post factum.
We don't even know why deep networks reach a performance plateau; just 10 years people used to blame local minima, but the current thinking is that this is not the point (when the perfomance plateaus, the gradients tend to stay large). This is such a basic question about deep networks, and we don't even know this.
Update: This is more or less the subject of Ali Rahimi's NIPS 2017 talk on machine learning as alchemy:

[This answer was entirely re-written in April 2017, so some of the comments below do not apply anymore.]


Question:
If symmetry conditions are met, FIR filters have a linear phase. This is not true for IIR filters.
However, for what applications is it bad to apply filters that do not have this property and what would be the negative effect?

Let me add the following graphic to the great answers already given, with the intention of a specific and clear answer to the question posed. The other answers detail what linear phase is, this details why it is important in one graphic:

When a filter has linear phase, then all the frequencies within that signal will be delayed the same amount in time (as described mathematically in Fat32's answer). When a filter has non-linear phase, individual frequencies or bands of frequencies within the spectrum of the signal are delayed different amounts in time.
Any signal can be decomposed (via Fourier Series) into separate frequency components. When the signal gets delayed through any channel (such as a filter), as long as all of those frequency components get delayed the same amount, the same signal (signal of interest, within the passband of the channel) will be recreated after the delay.
Consider a square wave, which through the Fourier Series Expansion is shown to be made up of an infinite number of odd harmonic frequencies.
In the graphic above I show the summation of the first three components. If these components are all delayed the same amount, the waveform of interest is intact when these components are summed. However, significant group delay distortion will result if each frequency component gets delayed a different amount in time.
The following may help give additional intuitive insight for those with some RF or analog background.
Consider an ideal lossless broadband delay line (such as approximated by a length of coaxial cable), which can pass wideband signals without distortion.
The transfer function of such a cable is shown in the graphic below, having a magnitude of 1 for all frequencies (given it is lossless) and a phase negatively increasing in direct linear proportion to frequency. The longer the cable, the steeper the slope of the phase, but in all cases "linear phase". This is also consistent with the equation for Group Delay, which is the negative derivative of phase with respect to frequency.
This makes sense; the phase delay of 1 Hz signal passing through a cable with a 1 second delay will be 360°, while a 2 Hz signal with the same delay will be 720°, etc...
Bringing this back to the digital world, $z^{-1}$ is the z-transform of a 1 sample delay (therefore a delay line), with a similar frequency response to what is shown, just in terms of H(z); a constant magnitude = 1 and a phase that goes linearly from $0$ to $-2\pi$ from f = 0 Hz to f = fs (the sampling rate).

The simplest mathematical explanation is that the a phase that is linear with frequency and a constant delay are Fourier Transform pairs. This is the shift property of the Fourier Transform. A constant time delay in time of $\tau$ seconds results in a linear phase in frequency $-\omega \tau$, where $\omega$ is the angular frequency axis in radians/sec:
$$\mathscr{F}\{g(t-\tau)\} = \int_{-\infty}^{\infty}g(t-\tau)e^{j\omega t}dt$$
$$u = t - \tau$$
$$\mathscr{F}\{g(u)\} = \int_{-\infty}^{\infty}g(u)e^{-j\omega (u+\tau)}du$$
$$ = e^{-j\omega \tau}\int_{-\infty}^{\infty}g(u)e^{-j\omega u}du$$
$$ = e^{-j\omega \tau}G(j\omega)$$
If this post was helpful, I provide more intuitive details such as this in online courses on DSP that are combined with live workshops. You can find more details on current course offerings here: DSP_coach.com