In the absence of a gold standard, assessment of clinimetric properties of dichotomous variables should include reporting of the proportions of positive agreement (ppos) and negative agreement (pneg). For example, for a patient considering whether or not to undergo elective surgery, ppos represents the probability that a second physician would concur with a recommendation to undergo surgery and pneg represents the probability that a second physician would concur with a recommendation not to undergo surgery. This article uses a conditional binomial distribution to derive the sampling distributions of ppos and pneg. The sampling distribution can be used as a basis for confidence intervals and hypothesis tests.