The first part of The Emperors New Clothes pointed out the fallacies of IEXs core claims of being Fair and Transparent. I
n part 2, I will explain several serious issues with IEXs recentpaper[1]that they published to demonstrate the execution quality of their exchange. Despite the use ofLaTeXsoftware and publishing the paper on SSRN, the paper combines legitimate statistical analysis with unsubstantiated marketing claims. Yet, like most of the rhetoric that emanates from the Investors Exchange, the conclusions seem to get accepted at face value, despite the marketing content.
The perspective of this analysis, like my previous note, is on IEXs value asexchange. Therefore, for this commentary, we will examine IEXs paper, based on how their statistical analysis applies todisplayed limit ordersand trades which interact with such orders.
The paper, written by IEX, produced data on block size, midpoint volume, hidden volume, queue size, trade markouts, price improvement, price discovery, and market stability. Of those, the only statistics that exclusively focus on displayed orders are queue size and price discovery. My contention is that the analysis on block size, midpoint volume, hidden volume and price improvement should be re-done to compare the dark component to other dark pools and to separate out the displayed component for comparison to the other exchanges. This is particularly vital with regard to price improvement, as I discussed inpart 1 of this series, due to the regrettable decision of IEX to conflate both non-marketable and marketable flow together.
Price Improvement
My analysis, which is exclusive to marketable orders, analyzed price improvement via the use of public 605 data filed by IEX and other market centers. This data shows, unlike the conclusion reached in the IEX paper, that, while good, IEXs execution quality for marketable limit orders was inferior to most of the other exchanges. According to public 605 data aggregated by BestXStats, IEXs April EQ, for all marketable orders, was 97.4, while Bats was 93, NYSE ARCA was 92.5 and Nasdaq was 91.3. I also showed that IEX was significantly inferior, on the same basis, to the largest dark pool (UBSA, whose EQ was 71.6) and concluded that overall execution quality of IEX is good, but not what they claimed. The difference between this data and what was shown in the paper, is that the paper was based on trades, without regard to the type of orders that created them. That was truly an unfortunate choice, and is an excellent example of why exchanges should not grade their own homework.
Rule 605 categorizes trades for market center execution quality as marketable or non-marketable, and, I believe this segregation was done for a good reason. Marketable orders, including both marketable limit and market orders are executable, at least in part, at the NBBO. Non-Marketable orders, including inside the quote, at the quote, and outside the quote orders, however, are not immediately executable and, therefore, have much lower fill rates on average. My contention is that the more proper analysis is to only look at marketable orders, for two reasons:
Bias– Without analyzing the unfilled orders (which would incorporate opportunity costs that IEX did not study), non-marketable orders should, of course, outperform marketable orders, since they are priced less aggressively. Put simply, midpoint IOC orders either execute at the midpoint or not at all, and dark pools, including IEX, attract a much higher percentage of those orders. The data is quite compelling on this point. According to BestXStats data, in April, IEX received 42.3 billion shares of inside-the-quote orders and executed just over 560 million: a fill rate of 1.3% and58%of their reported executed marketable shares. This compares to Bats, which executed 629 million shares of inside the quote orders with a fill rate of 0.7%, but, that quantity represented only7%of their total marketable executed shares. Thus, it is not surprising that including the inside-the-quote orders distorted the results.
Lack of Relevance to Exchange Analysis– As stated in part 1, exchange analysis should be related to either displayed orders or orders routed to exchanges to interact with displayed orders to focus on the price discovery process of exchanges. Non-Marketable IOC orders, however, arenotsent in response to displayed liquidity at the far side of the spread. Those orders are looking for midpoint fills and tend to be part of a sequential, probing routing strategy often by so-called dark aggregators. Marketable orders, however, are most often sent to exchanges based on the displayed liquidity available, often as part of a parallel routing strategy. Thus, the notion of comparing both types of orders is statistically invalid as they are sent for different reasons and in a different sequence.
Markouts
Before delving into the IEX markout analysis, I want to commend IEX for including markouts in the paper, as its use for describing adverse selection is a very important statistical tool. (one of the best explanations of this was byKCG[2]which shows the impact of being lower in the composite queue for a stock, not individual market queues). The goal of markout analysis is to provide a statistically valid way to measure the profit or loss incurred for fills against the orders being measured. That said, markouts providesomeindication of venue quality for institutions, but it is incomplete. Unfortunately, any serious analysis for institutions should also include both the fill probability of the order, and the market impact of posting itself, both of which are quite different for dark and lit orders as well as between venues.
To explain, consider a simple choice often faced by an algorithm or trader: how to post an order on the bid side of the market[3]. In this example, assume that the order needs to be filled since the client is expecting its completion, which is different from market makers or proprietary trading firms that have the luxury ofnot tradingwhen they dont consider the trade to be profitable. Thus, this simple choice equates to these questions:
- Whether to post the order in the dark, as a displayed order, or as a displayed order with a subset of the order shown with the rest in reserve (for my European readers, those are called iceberg orders)?
- Which exchange or dark pool to post the order on?
The determination of how to answer this question depends on the total order size being worked, but includes the following considerations:
- Fill probability of each type of order at each venue – If the order is unfilled, the trader should assume that they will have to cross the spread at the time the order is cancelled.
- Explicit cost of each type of order at each venue – While these are not always passed on directly, they do impact directly the commission levels that brokers charge as these costs impact profitability directly.
- Market impact of posting each type of order at each venue – One can assume that dark orders, until executed, leak no information, and therefore wont have impact, but posting a displayed order, does directly impact the market, whether or not it is executed. The magnitude of this impact directly increases the cost of completing the parent order, which is a reason why institutional algorithms often post more in the dark early in the life-cycle of an order.
Properly constructed algorithms constantly monitor these costs to make decisions about how and where to post. While I have not refreshed this analysis recently, in my experience orders posted on cheaper or inverted venues create more impact than on the traditional maker-taker exchanges. For heavily traded or less volatile securities, this is less of an issue, but the impact can be significant in many cases.
As a result, it is reasonable to conclude that a valid analysis of trade markouts should separate hidden from displayed orders, as they have very different attributes. In addition, dark orders can move with the NBBO, while displayed orders are fixed, which makes them more vulnerable to adverse moves. Additionally, IEXs hidden orders are designed to avoid immediate negative movement which occurs when quotes are stale. Thus, while IEX should absolutely get credit for their dark orders when analyzing dark orders, they should be ignored when looking at displayed order quality. It is also worth noting that their own analysis, even with the preponderance of dark orders, places IEX in the middle of the pack on a net basis and behind the inverted exchanges. In future commentaries, I will be introducing markout analysis that is exclusive to executions against displayed orders and will dig deeper into this issue.
Price Discovery
The price discovery section requires little comment, since IEXs data shows that, on a volume weighted basis they have the smallest marginal exchange contribution to the NBBO. Their discussion of queue size, however, ispure marketing, as they maketwo unsubstantiated claims.First, they claim that shorter queue sizes generally reflect less competition at the inside, which means that IEX and the inverted exchanges should be desirable for market participants who prioritize probability of execution for limit orders but provide no substantiating evidence. (Their claim is not unreasonable on its surface, considering theBattalio et al Limit Order study[4], which proves that exchanges with lower or negative take fees have higher fill rates on average. IEX, however, is far more expensive than the inverted exchanges (BX and BatsY) as well as EDGA and they present no data to suggest that investors should post on IEX rather than the end of the queue at those venues.)
Second, the paper asserts that joining a longer line has an accordingly lower probability of ?ll without showingany datato back up that claim. Well-constructed SORs look at the consolidated queue across all markets when making routing decisions and evaluates certainty of execution, market impact and costs of each potential combination of options. The SOR must also consider the specific scenario (e.g. is the market moving fast, is the order smaller or larger than the consolidated queue? is the client cost sensitive? etc.) Considering this, the question that needs to be answered is if IEX, by virtue of having smaller queue sizes, has a higher certainty of execution and I would suggest that is unlikely for three reasons.
First, larger venue queues create more certainty of execution. Many routing decisions are for order sizes smaller than the consolidated queue, and, in that case, when one venue has sufficient quantity to fill the entire order, routers often send the full order to that venue to increase fill certainty. In situations where an order needs to be routed to multiple exchanges, the lack of a fill could be extremely costly, making the certainty of the larger liquidity queues attractive. Second, while IEX is cheaper than the large maker-taker exchanges that have more posted liquidity, they are more expensive than the BX, BatsY and EDGA exchanges, which leaves them in an undesirable middle position when costs are evaluated. Third, the speedbump decreases the certainty of fill, since it makes IEXs quote slightly less reliable at any point in time. This was explained well byLarry Tabb in his recent post[5]. In sum, while further study is required to see how fill probability and adverse selection actually manifest at IEX, there is no reason to believe the assertions made in their study.
Lastly, readers should consider that queue sizes, despite being important on the most liquid stocks with 1 cent spreads, are less relevant for the majority of stocks that have wider spreads. This is because, when the NBBO is wider than 1 cent, an exchange can set a new, more aggressive price and, therefore, become first in the queue, at least until others join them. This type of action is the essence of price discovery, and should be a main consideration for issuers and investors. Unfortunately for IEX, according to data from MayStreet, IEX very rarely sets the NBBO. For example, in April, across all stocks that traded more than 1 million shares in a trading day, IEX set the NBBO less than 1% of the time, which was the lowest percentage for any exchange that traded in all those securities except for the CHX. Across all equity securities, IEX fared a little better, setting the NBBO 1.32% of the time, but that is still more than an order of magnitude less than Nasdaq at 32.15%, ARCA at 20.08%, and the NYSE (which is hampered by only trading in its own listings) at 19.9%.
In conclusion, from the perspective of displayed orders, the Emperors New Clothes analogy also holds for a purely statistical analysis of IEXs execution quality. Their paper, produced a couple of months ago, has not been questioned publicly to my knowledge, despite its flaws and lack of applicability to the exchange market. I can only hope that the industry will wake up to the inherent conflicts created by exchanges and brokers designing and publishing their own statistics and either demand independent verification or push the SEC to standardize metrics. Frankly, both of those are overdue, if we truly want the equity markets to be focused on best execution.
[1]https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2955297
[2]https://www.kcg.com/uploads/documents/KCG_NeedForSpeed1-Report.pdf
[3]I have simplified the scenario on purpose, but in the real world, there is also the option of routing to remove liquidity.
[4]Battalio, Corwin, & Jennings Dec 2013,https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2367462
