FileMaker Portal Sorting That Doesn’t Suck™
I am working on a FileMaker project that uses a “home” layout featuring a portal. The portal’s “parent” is a global text field containing a list of ID numbers. It’s related to an ID field in the child table. Nothing too unusual.
Then I started looking at sorting the portal. One nice thing about a list or table view is that it’s easy to give your users the ability to change the sort order on the fly. A portal’s sort order, though, is “baked in.” There’s really only one way to sort a portal: via the fields you specify in the portal’s Sort options in layout mode.
What if you want to click a column heading to sort? If you’ve used other dynamic portal-sorting techniques, you know they can be somewhat cumbersome. My thought was: I already have a list of child records (which I loop through in a script to populate my global text field). Why don’t I just sort these records before I loop through them? My ID list will then be in the order that I want.
Not so fast — the order of the IDs in the global field is ignored by the relationship. The records still display in an unsorted portal in the order they were created. Likewise, the global ID list can’t be used by the portal’s sort options.
I decided to try an experiment: what I needed was a field in the child table that calculates its position in the list of IDs. Then I could just sort the portal by that number.
Now, FileMaker lets us count the values in a list, filter values, and determine the position of a character. What it doesn’t provide is a function to tell you on what line in a list a certain value appears. That is what this calculation does:
Let( [
idList = ¶ & $$gIDs & ¶ ; // substitute your global var or field
id = ¶ & ID & ¶ ; // substitute your ID field
end = Position ( idList ; id ; 1; 1 ) ;
cutList = Left ( idList ; end ) ;
result = ValueCount ( cutList )
];
result )For example, let’s say our ID is “XYZ,” and it’s 4th in a list of 10 items. We first chop off all the text that follows XYZ (items 5-10). This leaves us with items 1-4 (“cutList” above), and XYZ becomes the last value in the list. Now its position in the list is the same as the number of values in the list: 4.
All that’s left to do is sort the portal by this field. It works!
To recap, you will need:
- A script that gathers the child IDs into a global variable or global text field, and does a sort.
- A calculated field in your child table that tells you where its ID is in the global list of IDs.
- A portal that is sorted by the calc field.
The great thing about this technique is that all the logic and control is script-driven, which makes it easy to set up and to make changes down the road. You will not be incurring technical debt.
This technique can be extended in all kinds of fun ways with conditional formatting, script triggers, etc. You can even allow your users to do a custom sort. Yes, a custom-sorted portal! See the example file for a basic implementation.
I would say that this technique lends itself best to an interface that is driven by a single-record/global ID list. I.e., we gather IDs as in step 1 above, not just to sort, but also to determine which records are displayed, using a relationship from a global ID field to the ID field in the child records. Otherwise, you’d need to trigger the sort as you navigate from record to record, and possibly do additional housekeeping to keep track of each record’s sort order. Multiple windows might also present challenges.
A few last points to keep in mind:
1. The calculation above should be set to Unstored.
2. If you use a global field (rather than a variable) to store the IDs, make sure its type is Text — a number field can’t store a list.
3. It seems, based on my limited testing, that when the calculated sort field is driven by a global field, the sort script does not require a window refresh step. When the calc references a global variable, it does require a refresh.
4. My friend and fellow FileMaker developer Matt Navarre pointed out that we don’t necessarily need to know the value number of the ID in the list — although you may want it for other purposes — it’s sufficient to simply know the order, which can be accomplished with the position function alone. Instead of a neat 1, 2, 3, 4, the result will be something like 1, 6, 11, 16, but gives us the same sort order:
Let ( [
idList = ¶ & $$gIDs & ¶ ; // substitute your global var or field
id = ¶ & ID & ¶ ; // substitute your ID field
result = Position ( idList ; id ; 1; 1 )
];
result )Download the sample file.
(Edit: 5/25/11 -added Freeze Window script step)
I look forward to your comments.
If you found this interesting, be sure to read the followup article.
Tags:
4. May 2011 at 11:43
Clever thinking!
I am a bit concerned about the performance with a lot of records, but then again, sorting a lot of child records is never fun, even when FM does it itself.
It will definitely be usefull for me, thx again.
4. May 2011 at 12:02
Hey there, Mr. Tom!
I’ve tried both of the other techniques and have found the “dynamic-portal-sorting” method to be too slow over a network, and the “hidden tab element/hidden-sorted-portals” technique is a lot of work (compounded when you want to add another sort field).
Thank you so very much for sharing this new technique and the demo file! Very clever! I look forward to implementing the “Fitch Portal Sorting Technique™” immediately! :D
P.S. Thanks also for the new (to me) term, “technical debt”. I will be having less of that now. ;)
4. May 2011 at 12:26
We know there’s more than one way to skin a calc in FM.
My personal approach. I’ve made this into a custom calc so that I can determine the index value for a selected item in any given list of separated values like a value list.
Let
(
[
items = $$gIDs ;
items = If ( Right (items ; 1 ) "¶" ; items & "¶" ; items )
];
ValueCount (
Left (
items ;
Position ( items ; ID ; 1; 1 ) + Length ( ID )
)
)
)
4. May 2011 at 16:49
As with the sorting burden, the following is not recommended for very large record sets.
Replace your current portalSort calc with the following to determine each record’s post-sort position:
valPos ( FoundList ( Child::ID ; 1 ); Child::ID )
The following custom functions will allow you to remove the ID-building loop entirely.
FoundList ( field; start )
// start should always be set to 1
GetNthRecord ( field ; start ) &
Case( start < Get ( FoundCount );
"¶" & FoundList ( field; start + 1 )
) &
Case( start = Get ( FoundCount ); "¶")
valPos ( array ; value )
PatternCount ( Left ( "¶" & array; Position ( "¶" & array; "¶" & value & "¶"; 1; 1)); "¶")
Example: valPos ( FoundList ( Child::ID ; 1 ); Child::ID )
5. May 2011 at 12:55
@rmw and @Audrey this method is not going to be the fastest, but IMO is acceptable on related sets of a reasonable size (<1,000). YMMV. Personally I don't think a portal is a good choice for presenting thousands, or even hundreds, of records.
@Rob and @Heather, I would encourage anyone to modify the calc in a way that seems most clear. I've seen a number of variations. In fact, I was going to include a "flattened" version of my calc in my original post but edited it out for simplicity. Here it is:
ValueCount ( Left ( ¶ & $$gIDs & ¶ ; Position ( ¶ & $$gIDs & ¶ ; ¶ & ID ; 1; 1 ) ) )Darren Terry and Chis Manton both pointed me to Sander Selover's custom function, which strikes me as very clean and readable:
Let ( [v = ¶ & _listOfValues & ¶ ;
p = Position ( v ; ¶ & _value & ¶ ; 1 ; 1 )
] ;
ValueCount ( Left ( v ; p ) )
)
(Notice I'm not using quotation marks around the paragraph symbols, it's not required and it's more readable, I think, without them.)
I would not expect such variations to impact the performance, and whether you put the calc into a custom function is again, really just a matter of taste and/or your organization's coding conventions.
@Heather I really appreciate your taking the time to post the FoundList custom function, but I have to tell you it does not perform well at all on large set of records. It was really noticeable on anything over a few hundred child records.
But again, for smaller related sets it's not going to make a perceptible difference, so if you prefer to put your logic in a custom function there's no reason not to. My preference these days is to put my logic into scripts, all else being equal.
For best performance in my example file:
5. May 2011 at 16:13
To prevent the list from being rebuilt by each record, the Found List custom function should be used to set the current list into a global that will be referenced instead; valPos ( Child::_gList; Child::ID ).
I don’t recall the details but thought I saw concern in someone’s post about using this technique in multi-user environments. This is not a problem if the portalSort calc is unstored and dependent on a global [or other unstored result like FoundList( )].
6. May 2011 at 09:29
@Heather, stashing the ID list in a global field or variable is the essence of this technique. That’s what I tested using your Found List function, and the results were in line with Jason Young’s excellent “What’s Faster” presentation from last year’s DevCon: loop w/freeze outperforms GetNth in building an ID list by a whopping margin.
Thanks for pointing out that this technique is multi-user friendly, I should have mentioned that in the article.
16. September 2011 at 09:05
Works awesome and thank you!
When I implemented it on one of our solutions, I was getting some screen flicker. Tweaking the GTRR to open in a new window at -1000 from the top and then closing that window when I was done eliminated it completely.
Thanks Again!
5. February 2012 at 06:33
I have used this technique in my solutions since I read this great article. Yesterday, as I was working on it in my new solution, suddenly something dawn on me.
Instead of using an un-stored calculation in the sort field, I created a regular sort field. Then in the script, after performed the record sort, I used the Replace Field Content to populate my sort field with custom value starting with 1 and increment by 1. In the portal, I just sort this regular sort field
This way, there is no need to use the custom function and there is no un-stored calc. I have tested this many times and it seems to work and it is fast too.
I am not sure whether this technique is good because I am new to FileMaker. Can someone tell me ?
5. February 2012 at 17:38
Thanks for writing. I’m glad you brought that up, and the answer is: that may be acceptable for a single-user solution, but it’s not multi-user friendly. What happens when two or more people try to sort some of the same records? Think about it.
Another potential problem is the speed: if there are a lot of records I think a Replace might end up being slower than an unstored calc.
Lastly, you are changing the modification timestamp of the records with each Sort, which may be undesirable.