Core Data Fetch requests slow on large data set
core-data, ios, large-files, nsfetchrequest, subquery
Solution
Here some hints to achieve what you want. Anyway, you always need to measure your changes to see if they are valid or not. It's a feedback process. Change and measure. Change and measure and so on.
- Prefetched relationships
- Use batches
- Prefer `beginwith` or `endswith` instead of `contains`
- Avoid [cd] when necessary
- Normalize strings
- ...
These are just hints.
I really suggest to take a look High Performance Core Data blog. There are a video, slides and code on how to achieve this. Matthew Morey did a very good job.
Also Apple Developer site (you must be a member to watch it), in WWDC 2013 Videos, has a session called Core Data Performance Optimization and Debugging (session 211) which talks about performance.
Problem
This is my first Core Data project, and I need advice on speeding up my fetch requests. My Core Data model contains 2 entities, `Wells` and `Fluids`. `Wells` has 50,000 records, and `Fluids` has 2 million records. They look like the following. ``` Wells nams relation wellsToFluids Fluids text1, text2, etc. relation fluidsToWells ``` Fetch requests on `Wells` are very fast. Fetch requests on `Wells` combined with data from `Fluids` accessed via compound predicates via the `wellsToFluids` relation are slow. And, I am seeing unexpected fetch times on different predicates. I am building a compound predicate, based on user selections. But basically the situation is the following Searches on `Wells`: ``` predicateWithFormat: @"(wellNumber == 1)" ``` .001 sec Searches on Fluids: ``` predicateWithFormat: @"(ANY wellsToFluids.text2 CONTAINS[c] stringToFind)" ``` 1.3 sec. (on Mac simulator – REALLY slow on iPhone or iPad) Fetch on `Wells` and `Fluids`: ``` predicateWithFormat: @"(wellNumber == 1) AND (ANY wellsToFluids.text1 CONTAINS[c] stringToFind)" ``` 3.2 sec. Fetch on `Wells` and multiple `Fluids` attributes: ``` predicateWithFormat: @"(wellNumber == 1) AND (ANY wellsToFluids.text1 CONTAINS[c] stringToFind) AND (ANY wellsToFluids.text2 CONTAINS[c] stringToFind)" ``` 6 sec. Changing `Fluids` portion of predicate to subquery yeld weird results. Fetch on only `Fluids` with subquery: ``` predicateWithFormat:@"(SUBQUERY(wellsToFluids, $x, ANY $x.text1 CONTAINS[c] stringToFind).@count !=0)" ``` 12 sec. Fetch on `Wells` and `Fluids` with subquery: ``` predicateWithFormat: @"(wellNumber == 1) AND (SUBQUERY(wellsToFluids, $x, ANY $x.text1 CONTAINS[c] stringToFind).@count !=0)" ``` 3.2 sec. Time does not change for additional attributes added in fetch from `Fluids`, stays at a fairly constant 3.2 sec. Also tried fetching from `Fluids`, then using relation back to `Wells` – but that did not improve speed. Can someone can give me some pointers on improving the fetch setup or it is just a feature of 2 million records?