Performing calculations on the add columns

[sasrsc]

I have a rather complex collection where I add in a bunch of new columns...
It's not so bad basically 

ClearCollect(CollectionName,AddColumns(ExistingTable as _existing,"My New Col1",somecode1,"My New Col2",somecode2))

What I want to do is perform calculations on these new columns...preferably when adding in the collection...at the time they are added. As the "somecode" for each column is pretty long, I really didn't want to repeat myself and I was hoping I could do this. But that doesn't appear to be the case...

If('My New Col1'="Mickey" && 'My New Col12'="Mouse","Awesome","Bummer")

Is there a way to do the above or must I either...
1) simply do it in another collection
2) re-type the whole code for the extra new column comparison - so this would seriously suffer some "DRY"

ClearCollect(CollectionName,AddColumns(ExistingTable as _existing,"My New Col1",somecode1,"My New Col2",somecode2,"GreatDay",If(somecode1 ="Mickey" and somecode2="Mouse","awesome","bummer")))

Here's the current full code, and my intent is that the new columns will have lengthy statements, so if that's the case it appears that I'll do another collection for simplicity's sake?

ClearCollect(
    WorkloadSnapshot,
    AddColumns(
        Attendees As _Att,
        "FlightArrivalDate",
        LookUp(
            EventFlightsArr,
            Email = _Att.Email || If(!IsBlank(WorkerNo) && WorkerNo <> "Guest", WorkerNo = _Att.SourceId) || (Trim(Upper(LastName)) in _Att.'Last Name' && Trim(Upper(FirstName)) in _Att.'First Name'),
            FlightArrivalDate
        ),
        "FlightDepartDate",
        LookUp(
            EventFlightsDep,
            Email = _Att.Email || If(!IsBlank(WorkerNo) && WorkerNo <> "Guest", WorkerNo = _Att.SourceId) || (Trim(Upper(LastName)) in _Att.'Last Name' && Trim(Upper(FirstName)) in _Att.'First Name'),
            FlightDepartDate
        ),
        "Hotel Bookings",
        LookUp(
            EventHotels,
            Email = _Att.Email || If(!IsBlank(WorkerNo) && WorkerNo <> "Guest", WorkerNo = _Att.SourceId) || (Trim(Upper(LastName)) in _Att.'Last Name' && Trim(Upper(FirstName)) in _Att.'First Name'),
            Bookings
        ),
        "HotelMinCheckIn",
        LookUp(
            EventHotels,
            Email = _Att.Email || If(!IsBlank(WorkerNo) && WorkerNo <> "Guest", WorkerNo = _Att.SourceId) || (Trim(Upper(LastName)) in _Att.'Last Name' && Trim(Upper(FirstName)) in _Att.'First Name'),
            MinCheckIn
        ),
        "HotelMaxCheckOut",
        LookUp(
            EventHotels,
            Email = _Att.Email || If(!IsBlank(WorkerNo) && WorkerNo <> "Guest", WorkerNo = _Att.SourceId) || (Trim(Upper(LastName)) in _Att.'Last Name' && Trim(Upper(FirstName)) in _Att.'First Name'),
            MaxCheckOut
        ),
        "HotelNtsBooked",
        LookUp(
            EventHotels,
            Email = _Att.Email || If(!IsBlank(WorkerNo) && WorkerNo <> "Guest", WorkerNo = _Att.SourceId) || (Trim(Upper(LastName)) in _Att.'Last Name' && Trim(Upper(FirstName)) in _Att.'First Name'),
            NtsBooked
        ),

        "HotelNtsExpected",
        LookUp(
            EventHotels,
            Email = _Att.Email || If(!IsBlank(WorkerNo) && WorkerNo <> "Guest", WorkerNo = _Att.SourceId) || (Trim(Upper(LastName)) in _Att.'Last Name' && Trim(Upper(FirstName)) in _Att.'First Name'),
            NtsExpected
        )
    )
);

[RandyHayes · Accepted solution]

@sasrsc 

Let's start with a Formula change...Please consider changing your Formula to the following:

ClearCollect(
    WorkloadSnapshot,
    ForAll(Attendees As _Att,
        With({_lookup: LookUp(
                           EventFlightsArr,
                           Email = _Att.Email || If(!IsBlank(WorkerNo) && WorkerNo <> "Guest", WorkerNo = _Att.SourceId) || (Trim(Upper(LastName)) in _Att.'Last Name' && Trim(Upper(FirstName)) in _Att.'First Name'
                       )},
            Patch(_Att, 
                {FlightArrivalDate: _lookup.FlightArrivalDate,
                 FlightDepartDate: _lookup.FlightDepartDate,
                 'Hotel Bookings': _lookup.Bookings,
                 HotelMinCheckIn: _lookup.MinCheckIn,
                 HotelMaxCheckOut: _lookup.MaxCheckOut,
                 HotelNtsBooked: _lookup.NtsBooked,
                 HotelNtsExpected: _lookup.NtsExpected
                }
            )
        )
    )
);

This will cut down on the length of your formula considerably and also reduce the number of Lookups that you are doing in the formula for performance.

Now...how about calculations?  This can be done by just reshaping the table level after level.

For example (and consider this in context of the above Formula).

Let's say you wanted to do some calculation on the Flight dates to get the amount of time from Arrival to Depart...

ClearCollect(
    WorkloadSnapshot,
    AddColumns
        ForAll(Attendees As _Att,
            With({_lookup: LookUp(
                               EventFlightsArr,
                               Email = _Att.Email || If(!IsBlank(WorkerNo) && WorkerNo <> "Guest", WorkerNo = _Att.SourceId) || (Trim(Upper(LastName)) in _Att.'Last Name' && Trim(Upper(FirstName)) in _Att.'First Name'
                           )},
                Patch(_Att, 
                    {FlightArrivalDate: _lookup.FlightArrivalDate,
                     FlightDepartDate: _lookup.FlightDepartDate,
                     'Hotel Bookings': _lookup.Bookings,
                     HotelMinCheckIn: _lookup.MinCheckIn,
                     HotelMaxCheckOut: _lookup.MaxCheckOut,
                     HotelNtsBooked: _lookup.NtsBooked,
                     HotelNtsExpected: _lookup.NtsExpected
                    }
                )
            )
        ),
        "TimeHere", DateDiff(FlightArrivalDate, FlightDepartDate, Days)
    )
);

You can see in the above, we just added another column to the results for the Table from the ForAll, and we did a calculation on it.

You can continue to "build" on the calculations with more AddColumns.

Ex.

ClearCollect(
    WorkloadSnapshot,
    AddColumns(
        AddColumns
            ForAll(Attendees As _Att,
                With({_lookup: LookUp(
                                   EventFlightsArr,
                                   Email = _Att.Email || If(!IsBlank(WorkerNo) && WorkerNo <> "Guest", WorkerNo = _Att.SourceId) || (Trim(Upper(LastName)) in _Att.'Last Name' && Trim(Upper(FirstName)) in _Att.'First Name'
                               )},
                    Patch(_Att, 
                        {FlightArrivalDate: _lookup.FlightArrivalDate,
                         FlightDepartDate: _lookup.FlightDepartDate,
                         'Hotel Bookings': _lookup.Bookings,
                         HotelMinCheckIn: _lookup.MinCheckIn,
                         HotelMaxCheckOut: _lookup.MaxCheckOut,
                         HotelNtsBooked: _lookup.NtsBooked,
                         HotelNtsExpected: _lookup.NtsExpected
                        }
                    )
                )
            ),
            "TimeHere", DateDiff(FlightArrivalDate, FlightDepartDate, Days)
        ),
        "someText", "You will be here " & Text(TimeHere) & " days and will checkin at " & HotelMinCheckIn &". We will see you depart " & FlightDepartDate
    )
);

I hope this is helpful for you.

[sasrsc · Accepted solution]

As usual - I am in awe of your wisdom. Thanks Randy.

[sasrsc]

I have a follow up question Randy as I have to deviate slightly here...
This is my revised version based on your input which works brilliantly...
As there are 3 different data sources and only the EventHotels involved multiple needs to lookup, I did the with for that one only and then did Lookups with the EventFlightsArr and EventFlightsDep. 

I don't like the Lookups piece because I prefer to reference something like this... 

Email = _Att.Email

Instead of like this...

Email = EventFlightsArr[@Email]

Is it possible to improve on this?

ClearCollect(
    WorkloadSnapshot,
    AddColumns(
        ForAll(
            Attendees As _Att,
            With(
                {
                    _EventHotels: LookUp(
                        EventHotels,
                        Email = _Att.Email || If(
                            !IsBlank(WorkerNo) && WorkerNo <> "Guest",
                            WorkerNo = _Att.SourceId
                        ) || (Trim(Upper(LastName)) in _Att.'Last Name' && Trim(Upper(FirstName)) in _Att.'First Name')
                    )
                },
                Patch(
                    _Att,
                    {
                        HotelMinCheckIn: _EventHotels.MinCheckIn,
                        HotelMaxCheckOut: _EventHotels.MaxCheckOut,
                        HotelNtsBooked: _EventHotels.NtsBooked,
                        HotelNtsExpected: _EventHotels.NtsExpected,
                        HotelBookings: _EventHotels.Bookings
                    }
                )
            )
        ),
        "FlightDepartDate",
        LookUp(
            EventFlightsDep,
            Email = EventFlightsDep[@Email] || If(
                !IsBlank(EventFlightsDep[@WorkerNo]) && EventFlightsDep[@WorkerNo] <> "Guest",
                SourceId = EventFlightsDep[@WorkerNo]
            ) || (Trim(Upper('Last Name')) in EventFlightsDep[@LastName] && Trim(Upper('First Name')) in EventFlightsDep[@FirstName]),
            FlightDepartDate
        ),
        "FlightArrivalDate",
        LookUp(
            EventFlightsArr,
            Email = EventFlightsArr[@Email] || If(
                !IsBlank(EventFlightsArr[@WorkerNo]) && EventFlightsArr[@WorkerNo] <> "Guest",
                SourceId = EventFlightsArr[@WorkerNo]
            ) || (Trim(Upper('Last Name')) in EventFlightsArr[@LastName] && Trim(Upper('First Name')) in EventFlightsArr[@FirstName]),
            FlightArrivalDate
        )
    )
);

[RandyHayes]

@sasrsc 

Have a go with this...

ClearCollect(
    WorkloadSnapshot,
    AddColumns(
        ForAll(
            Attendees As _Att,
            With(
                {
                    _EventHotels: LookUp(
                        EventHotels,
                        Email = _Att.Email || If(
                            !IsBlank(WorkerNo) && WorkerNo <> "Guest",
                            WorkerNo = _Att.SourceId
                        ) || (Trim(Upper(LastName)) in _Att.'Last Name' && Trim(Upper(FirstName)) in _Att.'First Name')
                    ),
                    
                    _EventFlightsDep:  LookUp(
		        EventFlightsDep As _departs,
		        _departs.Email = _Att.Email || 
		        If(!IsBlank(_departs.WorkerNo) && (_departs.@WorkerNo <> "Guest"), SourceId = _departs.WorkerNo) || 
		        (Trim(Upper('Last Name')) in =_departs.LastName && Trim(Upper('First Name')) in _dpearts.FirstName)		        
                   ),
                   
                   _EventFlightsArr:  LookUp(
                        EventFlightsArr As _arrives,
                        _arrives.Email = _Att.Email || 
                        If(!IsBlank(_arrives.WorkerNo) && (_arrives.WorkerNo <> "Guest"), SourceId = _arrives.WorkerNo) || 
                        (Trim(Upper('Last Name')) in _arrives.LastName && Trim(Upper('First Name')) in _arrives.FirstName)            
                   )
                },
                Patch(
                    _Att,
                    {
                        HotelMinCheckIn: _EventHotels.MinCheckIn,
                        HotelMaxCheckOut: _EventHotels.MaxCheckOut,
                        HotelNtsBooked: _EventHotels.NtsBooked,
                        HotelNtsExpected: _EventHotels.NtsExpected,
                        HotelBookings: _EventHotels.Bookings
                    }
                )
            )
        ),
        "FlightDepartDate", _EventFlightsDep.FlightDepartDate,
        "FlightArrivalDate", _EventFlightsArr.FlightArrivalDate
    )
);

[sasrsc]

Just because this is pretty cool, I'm sharing the revised code ... yet, to work on the error variables but now I have everything and it's cleaner than it was before and easier for the rest of my team to do the presentation layer in the gallery!

ClearCollect(
    WorkloadSnapshot,
    AddColumns(
        ForAll(
            Attendees As _Att,
            With(
                {
                    _EventHotels: LookUp(
                        EventHotels,
                        Email = _Att.Email || If(
                            !IsBlank(WorkerNo) && WorkerNo <> "Guest",
                            WorkerNo = _Att.SourceId
                        ) || (Trim(Upper(LastName)) in _Att.'Last Name' && Trim(Upper(FirstName)) in _Att.'First Name')
                    ),
                    _EventFlightsDep: LookUp(
                        EventFlightsDep As _departs,
                        _departs.Email = _Att.Email || If(
                            !IsBlank(_departs.WorkerNo) && _departs.WorkerNo <> "Guest",
                            _departs.WorkerNo = _Att.SourceId
                        ) || (Trim(Upper(_departs.LastName)) in _Att.'Last Name' && Trim(Upper(_departs.FirstName)) in _Att.'First Name')
                    ),
                    _EventFlightsArr: LookUp(
                        EventFlightsArr As _arrives,
                        _arrives.Email = _Att.Email || If(
                            !IsBlank(_arrives.WorkerNo) && _arrives.WorkerNo <> "Guest",
                            _arrives.WorkerNo = _Att.SourceId
                        ) || (Trim(Upper(_arrives.LastName)) in _Att.'Last Name' && Trim(Upper(_arrives.FirstName)) in _Att.'First Name')
                    )
                },
                Patch(
                    _Att,
                    {
                        HotelMinCheckIn: _EventHotels.MinCheckIn,
                        HotelMaxCheckOut: _EventHotels.MaxCheckOut,
                        HotelNtsBooked: _EventHotels.NtsBooked,
                        HotelNtsExpected: _EventHotels.NtsExpected,
                        HotelBookings: _EventHotels.Bookings,
                        FlightArrivalDate: _EventFlightsArr.FlightArrivalDate,
                        FlightDepartDate: _EventFlightsDep.FlightDepartDate
                    }
                ) // end of patch
            ) // end of with
        ), // end of forall
        /* add remaining columns for the gallery to better function */
        "isQcError",false,
        "isHotelError",false,
        "isAirError",false,
        "isAirInError",false,
        "isAirOutError",false       
    )
);

haring what I think is the

[RandyHayes]

@sasrsc 
Yes...I am all about the maintainability of formulas!  When another person has to revise something, the cleaner the better.  I also personally hate repeating any formula (like all the Lookups you had at first) - mostly because, if all of a sudden there was some other criteria that was needed for the Lookup...then you would have to adjust every one of them...that can get painful (especially if you accidentally miss one).

Anyway, I am glad to help out!